Staff Data Engineer
Posted on 3/22/2024
ASAPP

201-500 employees

Research-based artificial intelligence software provider
Company Overview
ASAPP exists to elevate human performance with AI Native® technology. They focus on addressing complex problems in environments with huge systemic inefficiencies and lots of data where real solutions create significant economic impact.
Robotics & Automation
AI & Machine Learning

Company Stage

Series C

Total Funding

$387.6M

Founded

2014

Headquarters

New York, New York

Growth & Insights
Headcount

6 month growth

-4%

1 year growth

-1%

2 year growth

5%
Locations
New York, NY, USA
Experience Level
Entry
Junior
Mid
Senior
Expert
Desired Skills
Microsoft Azure
Redshift
Python
Airflow
MySQL
Apache Flink
BigQuery
Apache Spark
SQL
Apache Kafka
Tableau
AWS
JIRA
Looker
Data Analysis
Snowflake
Google Cloud Platform
CategoriesNew
Data Engineering
Data Management
Data & Analytics
Requirements
  • 7+ years industry experience with clear examples of strategic technical problem solving and implementation
  • Expertise in at least one flavor of SQL. (We use Amazon Redshift, MySQL, Athena and Snowflake)
  • Strong experience with data warehousing (e.g. Snowflake (preferred), Redshift, BigQuery, or similar)
  • Experience with dimensional data modeling and schema design
  • Experience using developer-oriented data pipeline and workflow orchestration (e.g. Airflow (preferred), dbt, dagster or similar)
  • Experience with cloud computing services (AWS (preferred), GCP, Azure or similar)
  • Proficiency in a high-level programming language, especially in terms of reading and comprehending other developers’ code and intentions. (We use Python, Scala, and Go)
  • Deep technical knowledge of data exchange and serialization formats such as Protobuf, YAML, JSON, and XML
  • Familiarity with BI & Analytics tools (e.g. Looker, Tableau, Sisense, Sigma computing or similar)
  • Familiarity with streaming data technologies for low-latency data processing (e.g. Apache Spark/Flink, Apache Kafka, Snowpipe or similar)
  • Familiarity with Terraform, Kubernetes and Docker
  • Understanding of modern data storage formats and tools (e.g. parquet, Avro, Delta Lake)
  • Knowledge of modern data design and storage patterns (e.g. incremental updates, partitioning and segmentation, rebuilds and backfills)
  • Experience working at a startup preferred
  • Excellent communication skills - (Slack/Email/Documents)
  • Experienced with end user management & communication (cross team as well as external)
  • Must thrive in a fast paced environment and be able to work independently with urgency
  • Can work effectively remotely (able to be proactive about managing blockers, proactive on reaching out and asking questions, and participating in team activities)
  • Experienced in writing technical data design docs (pipeline design, dataflow, schema design)
  • Can scope and breakdown projects, communicate and collaborate progress and blockers effectively with your manager, team, and stakeholders
  • Good at task management & capacity tracking (JIRA (preferred))
Responsibilities
  • Lead the batch analytics team by providing the groundwork to modernize our data analytics architecture
  • Design and maintain our data warehouse to facilitate analysis across hundreds of systems events
  • Rethink and influence strategy and roadmap for building efficient data solutions and scalable data warehouses
  • Review code for style and correctness across the entire team
  • Write production-grade Redshift, Athena, Snowflake & Spark SQL queries
  • Manage and maintain Airflow ETL jobs
  • Test query logic against sample scenarios
  • Work across teams to gather requirements and understand reporting needs
  • Investigate metric discrepancies and data anomalies
  • Debug and optimize queries for other business units
  • Review schema changes across various engineering teams
  • Maintain high-quality documentation for our metrics and data feeds
  • Work with stakeholders in Data Infrastructure, Engineering, Product and Customer Strategy to assist with data-related technical issues and build scalable cross platform reporting framework
  • Participate in, and co-manage our on-call rotation to keep production pipelines up and running