Full-Time
Data Engineer
Confirmed live in the last 24 hours
Prophecy
51-200 employees
Low-code platform for data engineering workflows
Entry, Junior
Palo Alto, CA, USA
Occasional travel to the office.
- Experience in Apache Spark, and you can write Python or Scala code
- Experience in complex data engineering (ETL) and have solved many performance issues with Spark jobs
- Experience in a technical customer-facing role - preferably solution engineering or wanting to do such a role
- Bachelor in CS (preferred)
- Be a resident expert in data engineering using Prophecy with Spark & Airflow
- Solve complex problems and educate
- Address complex customer issues related to Apache Spark and Data Warehouses
- Build automated tooling to solve repeatable problems
- Demonstrate ownership to drive to solution, be pro-active
- Participate in on-call rotation
- Help our customers (Fortune 500) onboard to the product, answering queries about Prophecy, Spark, and Airflow
- Provide best practices and guidance around pipeline runtime performance, usage of third-party libraries, CI/CD, and scheduling
- Build reusable Spark, SQL, and Airflow packages that solve repeated problems in data engineering (e.g., data quality, auditing, and connectors)
- Write high-performance Spark code
Prophecy.io offers a low-code platform that simplifies the creation and management of data workflows and schedules, primarily using Apache Spark and Apache Airflow. Users can develop Spark workflows and manage Airflow schedules with minimal coding, making it accessible for data engineers and enterprises. The platform also includes features for searching metadata and tracking data lineage at the column level, which helps users understand the flow and transformation of their data. Unlike many competitors, Prophecy.io focuses on providing a comprehensive suite of tools that enhance productivity and efficiency in data operations without requiring extensive coding knowledge. The goal of Prophecy.io is to empower medium to large enterprises to optimize their data processing capabilities through a subscription-based service that offers these low-code tools.
Company Stage
Series B
Total Funding
$67.6M
Headquarters
San Francisco, California
Founded
N/A
6 month growth
↑ 0%1 year growth
↑ 28%2 year growth
↑ 141%
Simplify's Take
What believers are saying
- The recent $35 million Series B funding round indicates strong investor confidence and potential for growth and innovation.
- Prophecy's generative AI assistant and low-code SQL capabilities democratize data access, empowering both technical and non-technical users to efficiently manage data pipelines.
- Strategic partnerships and sponsorships, such as with the Data + AI Summit, enhance Prophecy's visibility and credibility in the data engineering market.
What critics are saying
- The rapid evolution of data engineering tools could outpace Prophecy's current offerings, necessitating continuous innovation to maintain competitiveness.
- Reliance on platforms like Databricks for integration may limit Prophecy's flexibility and adaptability to other emerging technologies.
What makes Prophecy unique
- Prophecy.io's low-code platform uniquely combines Apache Spark and Apache Airflow, streamlining data engineering processes for enterprises with minimal coding effort.
- The introduction of the Data Transformation Copilot for Databricks positions Prophecy as a pioneer in leveraging generative AI for data preparation, setting it apart from traditional ETL tools.
- Prophecy's focus on metadata management and column-level lineage tracking offers enterprises enhanced data governance capabilities, distinguishing it from competitors.
Help us improve and share your feedback! Did you find this helpful?