AI/ML Engineer

About Prophecy 

The leader in AI-native data preparation and analysis, Prophecy is revolutionizing how the world’s top enterprises turn data chaos into reliable insights. We introduce the AI-native data lifecycle (generate, refine, deploy) where our industry leading AI agents and humans work hand-in-hand in visual and document interfaces to analyze, transform and prepare data, to ship trusted insights at enterprise scale. To learn more, visit us on LinkedIn.

---

The Role

You'll own the intelligence layer at Prophecy: the AI systems that let our agents understand data schemas, generate correct transformations, and reason over enterprise knowledge. This isn't a role where you implement someone else's spec — you make the calls on agent architecture, model selection, quality evaluation, and build vs. buy. The scope scales with you; strong engineers have enormous leverage here regardless of title.

---

What You'll Own

Agent design and quality. Architect multi-step AI agents end-to-end — problem decomposition, context retrieval, error recovery, output quality. Set the bar, build the evals, own the improvement loop.

Retrieval and knowledge systems. Own the RAG pipelines, vector and graph-based retrieval, and knowledge representations that let agents reason accurately over customer schemas, lineage, and semantics.

Code generation and data transformation. Own the AI layer behind Prophecy's core capability — natural language to production-grade transformations: prompt design, model selection, fine-tuning, output validation.

Production AI. Make AI that works in the lab work at scale — latency, throughput, observability, knowing when to optimize a model vs. restructure a system.

---

Who You Are

You've built coding agents or AI systems in production and owned them when things break. You think in failure modes and evaluation strategy, not capability demos. You make architectural decisions under uncertainty and know when to move fast versus when to do it right.

You're a builder who loves picking up new technologies — in a field where last year's best practice may already be obsolete, that instinct matters. You have a strong perspective on how coding agents should work and bring it to design discussions, not just implementation.

Experience level is flexible; what matters is the track record. We'd expect 4–6+ years in software engineering with meaningful AI/ML depth.

---

Stack and Background We Value

Python is the primary language. Retrieval infrastructure spans vector databases and knowledge graphs (Neo4j), backed by AWS and Kubernetes. Scala and Go are meaningful plusses — our language compiler and orchestrator are built on them. Our platform integrates with Databricks, Snowflake, BigQuery, DuckDB, and Spark; experience with any of these, or with data processing and analytics broadly, is genuinely valuable.

---

Location: San Mateo office, 2–3 days/week

Compensation: $250,000–$350,000 base + equity, calibrated to level.

Benefits and Perks 

• Prophecy covers 99% of employee health insurance premiums and 75% for dependents
• We offer $200 per month towards remote work, wellness, gyms, massages, facials, and more!
• Flexible PTO
• Professional development allowance
• Company sponsored Long Term Disability and Life Insurance, FSA/HSA, dental, vision, and more
• Ability to have your fingerprint on an innovative platform
• End-to-end ownership of your projects. 

*Benefits and perks may vary per country

Prophecy is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, colour, religion, sex, national origin, age, disability, or veteran status.