Lead Data Engineer
P2376
Posted on 2/21/2023
INACTIVE
Locations
Northbrook, IL, USA • Remote • Chicago, IL, USA • Portland, OR...
Experience Level
Entry
Junior
Mid
Senior
Expert
Desired Skills
Agile
Apache Spark
Java
Airflow
Microsoft Azure
Snowflake
SQL
Python
Requirements
- 5+ years proven ability of professional data development experience
- 5+ years proven ability of developing with HDFS/RDBS/Cloud Database Technologies
- 3+ years developing experience with SQL and either Java or Python
- 3+ years experience with PySpark/Spark
- 3+ years experience with Data Automation - e.g. Airflow, Azure Data Factory
- Data quality and testing best practices
- Full understanding of ETL concepts
- Strong understanding of CI/CD methodology
- Understanding of Object-Oriented and SOLID principles
- Strong understanding of Agile Principles
Responsibilities
- Lead the design of and develop Cloud and Data based solutions (Databricks, Snowflake, etc.)
- Perform unit and integration testing
- Participate in implementation of BI visualizations
- Collaborate with architecture and lead engineers to ensure consistent development practices
- Provide mentoring to junior engineers
- Participate in retrospective reviews
- Participate in the estimation process for new work and releases
- Collaborate with other engineers to solve and bring new perspectives to complex problems
- Drive improvements in people, practices, and procedures
- Embrace new technologies and an ever-changing environment
Desired Qualifications
- New Technical Product Development
- Cloud Engineering Experience (Azure)
- Databricks
- Snowflake
- Terraform
- Exposure to API development
- Exposure to Service Oriented Architecture
- Proficient with relational data modeling and/or data mesh principles
Retail data science, insights, and media platform