Senior AI Engineer - Testing

About Glasslake

Glasslake is building the operating system for SAP delivery — the infrastructure layer underneath the most complex and expensive IT projects companies run. SAP implementations move billions of dollars and thousands of consultants, but the knowledge generated in these projects is never captured: it lives in people’s heads, disappears when the project ends, and gets rebuilt from scratch on every new engagement.

Glasslake solves this with an agentic orchestration platform that coordinates every phase of SAP transformation, capturing decisions, tests, and context as a byproduct of work already underway — building an organizational memory that accumulates across projects instead of resetting.

We’re a team based in San Francisco and São Paulo, moving fast and already deployed with some of the largest SAP integrators in the world, including Delaware Consulting, KPMG, and NTT Data. The team comes from Stanford, Meta, Two Sigma, Microsoft, and NASA.

The Opportunity

You’ll own some of the hardest problems in applied AI: making agents reliable, fast, and trustworthy on enterprise systems where being wrong is expensive.

Testing is one of the most critical and labor-intensive parts of any SAP transformation. Our AI agents plan, execute, and validate tests of business processes inside real customer SAP landscapes, work that has traditionally taken teams of consultants weeks to do by hand.

Making agents perform at that level is an open frontier. You’ll take on problems with no off-the-shelf answers, explore them, define what success looks like, and carry the solution all the way to production with real customers.

What You’ll Do

Explore and define the problem

• Frame the problem. Turn observed agent behaviors into a clear problem statement, backed by evidence and your own experiments.

• Define success. Set the metrics, benchmarks, and evals that measure agent performance.

Build and own the solution

• Ship end to end. Design and ship agent and LLM features, from prompts and orchestration to the infrastructure behind them.

• Own the results. Watch how your work performs in production and keep improving it.

Work with experts and raise the bar

• Validate with experts. Work with SAP domain experts and present findings and decisions clearly.

• Raise the bar. Review code and designs, and help shape the team’s AI engineering practices.

What We’re Looking For

• 6+ years of engineering experience, including building AI agents or LLM features from scratch and shipping them to production with full ownership.

• A research-like approach. You set hypotheses, run experiments, and verify results before you build.

• Evaluation rigor. Experience measuring agent performance through evals, benchmarks, or regression suites, and acting on what they show.

• Strong Python and SQL, with hands-on experience with LLM APIs and agent frameworks.

• Infrastructure ownership. Comfortable running the workflow orchestration and cloud services behind your features.

• Production investigation. You query databases and read logs and traces to find out what actually happened.

• Self-direction and clear communication, including with domain experts who aren’t engineers.

• AI native. You use AI tooling daily as a force multiplier.

Nice to have

• Experience with computer-use or browser-automation agents.

• Software testing background, such as test automation or QA for complex systems, or familiarity with SAP testing.

• Familiarity with enterprise software ecosystems (SAP, ERP, CRM).

• Ability to work in TypeScript when a feature needs it.

• Early-stage startup experience where you owned a problem with no playbook.

• Experience mentoring engineers or acting as a technical reference on a team.

Why Join Glasslake Now

• Own open-ended agent problems end to end, from the first exploration to results in production.

• Time to explore a problem and test ideas before committing to a solution.

• Your agents run inside multi-million dollar transformations at the largest SAP integrators in the world, in a category being defined right now.

• Direct access to the founding team, customers, and SAP domain experts.