Facebook pixel

Senior Data Engineer
Data Systems
Confirmed live in the last 24 hours
Locations
Remote • United States
Experience Level
Entry
Junior
Mid
Senior
Expert
Desired Skills
Apache Spark
AWS
Apache Kafka
Data Analysis
Git
Airflow
Requirements
  • Four-year degree in Computer Science, or related field OR equivalent experience
  • Progressive experience in all of the following areas:
  • Designing frameworks and writing efficient data pipelines, including batches and real time streams
  • Understanding of data strategies, articulate data analysis & data model design and evolve data products according to business requirements
  • Experience with the Spark Ecosystem (YARN, Executors, Livy, etc)
  • Experience in large scale data streaming, particularly Kafka or similar technologies (Pulsar, Kinesis, etc)
  • Experience with data orchestration frameworks, particularly Airflow or similar
  • Experience with columnar data stores, particularly Parquet and Clickhouse
  • Strong SDLC principles (CI/CD, Unit Testing, git, etc)
  • General understanding of AWS EMR, EC2, S3
  • Experience with
  • “Lakehouse” technologies, particularly Iceberg or similar (DeltaLake, Hudi, etc)
  • Terraform for AWS
  • Astronomer
  • Open Source contribution experience
Responsibilities
  • Build out all aspects of the Demandbase Data ecosystem and move products from R&D into production scale
  • Design and build data pipelines to create the next generation of Demandbase's Unified Data Platform
  • Work across the data stack to build and productionalize data pipelines for massive amounts of data
  • Build DAGs in Airflow for orchestration and monitoring of data pipelines
Demandbase

501-1,000 employees

Company Overview
Demandbase is a targeting and personalization platform for business-to-business companies.