Facebook pixel

Senior Data Engineer
P2695
Posted on 5/17/2022
INACTIVE
Locations
Northbrook, IL, USA • Remote • Chicago, IL, USA • New York, NY...
Experience Level
Entry
Junior
Mid
Senior
Expert
Desired Skills
Agile
Apache Spark
Data Analysis
Development Operations (DevOps)
Hadoop
Business Analytics
Git
Java
Management
Microsoft Azure
Oracle
Scala
SCRUM
SQL
Python
Requirements
  • 5+ years proven ability of professional Data Development experience
  • 3+ years proven ability of developing with Hadoop/HDFS and SQL (Oracle, SQL Server)
  • 3+ years of experience with PySpark/Spark
  • 3+ years of experience developing with either Python, Java, or Scala
  • Full understanding of ETL concepts and Data Warehousing concepts
  • Exposure to VCS (Git, SVN)
  • Strong understanding of Agile Principles (Scrum)
  • Bachelor's Degree (Computer Science, Management Information Systems, Mathematics, Business Analytics, or STEM)
  • Cloud Environment Experience (Preferably Azure)
  • CI/CD Pipelines (Continuous Integration/Continuous Delivery)
  • TDD (Test Driven Development) Principles
  • ETL Principles (Extract Transform Load)
  • ADF (Azure Data Factory)
  • Version Control (GitHub or Azure DevOps)
  • Databricks
  • Databricks Notebooks
  • Python
  • PySpark
  • PyTest
  • Hadoop
Responsibilities
  • Participate in design and development of Hadoop and Cloud-based solutions
  • Perform unit and integration testing
  • Participate in implementation of BI visualizations
  • Collaborate with architecture and lead engineers to ensure consistent development practices
  • Provide mentoring to junior engineers
  • Participate in retrospective reviews
  • Participate in the estimation process for new work and releases
  • Collaborate with other engineers to solve and bring new perspectives to complex problems
  • Drive improvements in people, practices, and procedures
  • Embrace new technologies and an ever-changing environment
84.51 Degrees