Technical Program Manager
Preference Model
Technical Program Manager
Preference Model
Reinforcement learning environments for frontier AI
Preference Model is a superintelligence data research company. We build RL environments for training capable, better-aligned superintelligences.
Almost every aspect of how a model behaves is shaped by the reward signals it's trained on. The biggest problem with AIs today is that they don't always do what we intend. Sometimes that's because they aren't capable enough, and sometimes it's because they aren't aligned. Both problems come down to the training objectives.
We're trying to address both of these issues in the most direct, highest leverage way available to us, which today means researching how to build RL environments better, not just for the AIs of today but for the AIs as they will be when they outsmart us.
Over the past year, we've built RL environments for several frontier labs, and we're backed by $16M in seed funding led by a16z. Our founding team has previous experience on Anthropic’s data team building data infrastructure, and datasets behind Claude.
We're hiring our first Technical Program Manager, GTM. Frontier labs are deciding which RL environment partners to trust with their training runs, and the relationships we build with them now will shape much of where Preference Model goes next. Until now, those relationships have been run by our leadership alongside everything else. You'll be the first person whose full-time job is to own them.
This is a founding role with broad ownership. You will own communication and the relationship with our lab customers across the whole cycle: selling, delivery, scope expansion and contract negotiation. Our platform team owns the technical side of delivery. You'll have a real say in which labs we pursue, how we price our work, and what we build next.
You will help build our GTM playbook and eventually have the opportunity to expand the team.
You will report directly to a co-founder and work onsite in San Francisco alongside leadership, our capabilities team, and our platform team.
Success in this role means projects ship on timelines both sides trust, and the relationships you own are growing into larger engagements.
Own relationships with frontier lab customers. Be their main point of contact, push each relationship forward and grow accounts into larger engagements.
Decide with leadership which labs and markets to pursue, and where to focus as the market for RL environments changes.
Run the sales cycle: send samples, learn what each lab needs, and turn early conversations into scoped projects.
Shape pricing and packaging, and negotiate contracts with our co-founders.
Build feedback loops that translate customer feedback into focus areas and priorities.
Keep delivery on track with the internal team: set realistic timelines, review purchase orders, and partner with the platform team, which owns technical delivery, so nothing falls between teams.
Represent Preference Model publicly through writing, events and the AI community.
Build our go-to-market function from scratch: how we sell, scope, contract and deliver, and the playbooks that make it repeatable.
0 to 1 experience: You've built something from the ground up: at an early-stage startup, as a founder, or in VC.
A technical background: a computer science or closely related technical degree, or previous software engineering experience.
External customer-facing experience: for example as a forward deployed engineer, in customer success or operations, as a product manager, or in VC.
2+ years of relevant experience.
Experience negotiating commercial terms: customer contracts, pricing, or investment deals.
Confidence with senior, technical stakeholders and the follow-through to keep many relationships moving at once. You can push back on unrealistic asks, deliver bad news early, and ensure nothing gets dropped.
Nice to Have:
Experience with LLM training or post training strongly preferred.
Experience at a frontier AI lab, neo lab, or a neocloud.
Experience at a company known for strong customer support
Familiarity with RL environments, evals, benchmarks, or post-training data (RLHF, RLVR).
You write publicly (a blog, newsletter, or essays) or are active on social media. This is a plus.
You started as a software engineer or forward deployed engineer and have since taken on roles that pair technical depth with owning customer outcomes.
You have experience with LLM training or post training
You founded a company and sold to technical buyers yourself.
You've run engagements for frontier lab customers at an AI data, evals, or RL environment company.
Competitive cash and equity compensation
Ownership and autonomy in a fast moving startup environment
Opportunity to work with top Machine Learning Engineers
Health, Vision, Dental, benefits
401K match
Lunch provided daily, and a weekly snack order
We value diverse perspectives and experiences. If you're excited about this role but don't check every box, we still encourage you to apply.
Reinforcement learning environments for frontier AI
Preference Model builds reinforcement learning environments and harnesses to train frontier AI models, focusing on how reward signals shape model behavior. It designs robust RL harnesses and simulators that treat reward functions and graders as an attack surface, testing them across millions of training rollouts to harden against loopholes and gaming, with fail-safes for safety. Unlike others that only build models or tools, it targets the reward design and alignment bottlenecks, drawing talent from Anthropic, Stripe, Google DeepMind, and others, and backed by notable investors. Its goal is to broaden access to capable AI while preserving strong safeguards to keep frontier systems safe.
Explore more opportunities at Preference Model