As a Lead Software Engineer with a CI/CD and DevOps focus, you will have the opportunity to build a product that automates away the setup of CI/CD pipelines for our Data Science and Data Engineering teams. Our team uses Databricks, Github, Github actions, Terraform, Python, and Spark to accomplish these goals. We work alongside product owners, data scientists, and other engineers to bring this vision to life.

Responsibilities:
Take ownership of the design, features, and development of this solution. Some examples include:

• Lead the design and development of our automated CI/CD product in Databricks and Azure
• Perform unit and integration testing
• Understanding of infrastructure as code
• Collaborate with architecture, lead engineers, agilists, and product owners to ensure consistent development practices
• Provide mentoring to junior engineers
• Participate in retrospective reviews
• Participate in the estimation process for new work and releases
• Drive improvements in people, practices, and procedures
• Embrace new technologies and an ever-changing environment
• Meet with vendors and third parties to ensure we’re using the right tools and technologies

Requirements:
Bachelor’s degree typically in Computer Science, Management Information Systems, Mathematics, Business Analytics or another STEM degree.

• 5+ years proven ability of professional data development experience
• 3+ years proven ability of developing with Hadoop/HDFS
• 3+ years developing experience with either Java or Python
• 3+ years experience with PySpark/Spark
• Full understanding of ETL concepts
• Exposure to VCS (Git, SVN)
• Strong understanding of Agile Principles (Scrum) and CI/CD

 Preferred Skills – Experience in the following:

• Python
• Spark
• Docker
• Azure
• Databricks
• Terraform
• APIs
• RabbitMQ
• Github/Github Actions
• CI/C

#LI-DOLF #LI-REMOTE