Full-Time

AI/ML Agentic-Workflows Engineer – Sr/Staff

Updated on 4/16/2025

Prophecy

Prophecy

51-200 employees

Low-code platform for data engineering workflows

Compensation Overview

$200k - $500k/yr

Senior, Expert

San Francisco, CA, USA

This position is hybrid, indicating a mix of remote and in-office work.

Category
Applied Machine Learning
Natural Language Processing (NLP)
AI & Machine Learning
Required Skills
LLM
Machine Learning
AWS
Natural Language Processing (NLP)
LangChain
Data Analysis
Requirements
  • Proven success delivering AI and agentic products from concept to adoption, with hands-on design of real-time ML systems.
  • Experience with GenAI, LLMs, NLP, RAG, and Knowledge Graphs across the ML lifecycle (train, optimize, deploy, monitor).
  • Expertise in multi-agent systems and frameworks like LlamaGraph or LangChain for complex workflows.
  • Proficiency deploying and scaling ML/agentic solutions in cloud environments (e.g., AWS).
  • Passion for rapid iteration and high-quality execution with an entrepreneurial mindset.
  • BS in CS or equivalent, 5-10 years of experience; MS/PhD preferred.
Responsibilities
  • Provide hands-on technical leadership to design, develop, optimize and deploy Generative AI features, leveraging expertise in LLMs, RAGs, NLP, and AI infrastructure to optimize Agents quality, reliability and performance.
  • Support and Lead a strong technical team across the US and Bangalore, guiding them on state-of-the-art approaches to ML system design, Agent design and orchestration while establishing the best in class data-driven development practices.
  • Build agents that streamline data extraction, transformations and analysis.
  • Define scalable evaluation strategies that balance AI with human reviewers.
Desired Qualifications
  • ML/LLM work in code generation (e.g., Codex, text-to-SQL), semantic extraction, or knowledge graphs (e.g., Neo4j, Neptune).
  • Experience with big-data engines like Spark.
  • Compiler development for languages like SQL, Python, or Scala.
  • Optimization of ML models for low-latency, high-throughput production use.
  • Contributions to open-source AI/ML projects (e.g., Hugging Face, PyTorch).
  • Expertise in retrieval systems or vector databases (e.g., Pinecone, Weaviate).
  • Skill in evaluating tech and driving build/buy decisions.

Prophecy.io offers a low-code platform that simplifies the creation and management of data workflows and schedules, primarily using Apache Spark and Apache Airflow. Users can develop data processing tasks with minimal coding, allowing data engineers to focus on optimizing their operations rather than getting bogged down in complex programming. The platform includes features for metadata search and tracking the lineage of data at the column level, which helps users understand the flow and transformation of their data. Unlike many competitors, Prophecy.io emphasizes ease of use and accessibility for medium to large enterprises, providing a subscription-based service that grants access to its suite of tools. The goal of Prophecy.io is to enhance productivity and efficiency in data operations, making it easier for organizations to manage their data workflows.

Company Size

51-200

Company Stage

Series B

Total Funding

$116.5M

Headquarters

San Francisco, California

Founded

2017

Simplify Jobs

Simplify's Take

What believers are saying

  • Increased adoption of low-code platforms boosts Prophecy's market potential.
  • Generative AI integration aligns with industry trends for efficient data processing.
  • Rising demand for self-service data tools supports Prophecy's platform offerings.

What critics are saying

  • Competition from Alteryx and Talend could impact Prophecy's market share.
  • Rapid AI evolution may outpace Prophecy's current solutions.
  • Reliance on Databricks integration poses risks if platform changes unfavorably.

What makes Prophecy unique

  • Prophecy offers a low-code platform for data engineering, reducing development time and costs.
  • The platform integrates generative AI for efficient data preparation and transformation.
  • Prophecy provides strong data governance and lineage tracking capabilities for quality and compliance.

Help us improve and share your feedback! Did you find this helpful?

Benefits

Health Insurance

Unlimited Paid Time Off

Wellness Program

Professional Development Budget

Company Equity

Life Insurance

FSA/HSA

Long Term Disability

Growth & Insights and Company News

Headcount

6 month growth

3%

1 year growth

0%

2 year growth

12%
Big Data Wire
Mar 28th, 2025
Prophecy 4.0 Offers Fully Governed Self-Service Data Prep for Databricks SQL

Additionally, Prophecy has introduced built-in automation with a drag-and-drop interface.

Prophecy
Jan 20th, 2025
Prophecy takes in $47M to scale-up and re-imagine data integration with AI

I’m pleased to share that Prophecy today announced a $47M Series B1 round. Smith Point Capital led the round, with HSBC joining as a new investor and participation from existing investors including Berkeley SkyDeck, DallasVC, Insight Partners, JPMorgan Chase and SignalFire.

Finsmes
Jan 16th, 2025
Prophecy Raises $47M in Series B Extension Funding

Prophecy raises $47M in Series B extension funding.

SiliconANGLE
Jan 16th, 2025
Prophecy raises $47M for AI data pipelines

Prophecy Inc., a data copilot startup, raised a $47M Series B extension led by Smith Point Capital, with participation from HSBC and others. The company uses generative AI to automate data pipeline development, making data accessible across systems. Prophecy's copilot, integrated with Databricks, accelerates AI initiatives by simplifying data preparation. The funding will enhance its platform and expand its customer base by 2025.

MRWeb
Jan 16th, 2025
Funds for Data Integration 'Copilot' Firm Prophecy

San Diego, CA-based data integration software company Prophecy has raised $47m in a Series B1 round of funding, and will use the money to expand operations and its business reach.