Principal Software Engineer
AWS/Airflow/Devops
Posted on 7/19/2023
INACTIVE
Egen

201-500 employees

Cloud engineering & data platforms
Company Overview
Egen are cloud engineering and data experts partnering with fast-scaling startups and large enterprises to launch new digital applications, modernize data infrastructures, and create conversational interfaces powered by machine learning and AI.
Consulting

Company Stage

Pre-seed

Total Funding

$1.3M

Founded

2000

Headquarters

Naperville, Illinois

Growth & Insights
Headcount

6 month growth

0%

1 year growth

-1%

2 year growth

3%
Locations
Naperville, IL, USA • Remote in USA
Experience Level
Entry
Junior
Mid
Senior
Expert
Desired Skills
Kubernetes
Microsoft Azure
Python
Airflow
MySQL
Git
Postgres
AWS
Go
Terraform
Development Operations (DevOps)
Linux/Unix
CategoriesNew
DevOps & Infrastructure
Software Engineering
Requirements
  • Airflow expert with experience in setting up and managing MWAA on AWS
  • In-depth knowledge of orchestration concepts, including DAG (Directed Acyclic Graph) creation, task definition, operators, and best practices for workflow management
  • Proficiency in Python programming, with a focus on developing Airflow DAGs and custom operators
  • Strong understanding of Terraform, including the ability to write Terraform code for infrastructure as code (IaC) management
  • Familiarity with AWS services such as S3, Lambda, CloudWatch, Fargate, VPC, security groups, and other related technologies
  • Understanding of AWS Postgres and experience working with databases in the cloud environment
  • Basic-level knowledge of Unix shell scripting for basic automation and maintenance tasks
  • System-level architecture and design and implementation experience
  • Experience with Python/Golang and cloud SDKs like AWS SDK, Kubernetes clients library, Helm
  • Experience with Mysql, PostgreSQL, Bitbucket, GitHub
  • Knowledge of service mesh frameworks like Istio, Linkerd, and Hashicorp Consul Connect
  • Knowledge of IAM and JWT authorization, SSO/OAuth (Azure AD, Okta)
  • Knowledgeable of 12-factor app methodology and how they should be applied
Responsibilities
  • As a Principal Software Engineer at Egen you will provide technical leadership and expertise to a modernization project
  • You provide technical expertise to migrate legacy Control M (xml) jobs to run seamlessly through Airflow DAGs
  • You will design and scope delivery of the new MWAA (Managed Workflows for Apache Airflow) environment on AWS and recommend and define the roadmap for all teams involved
  • You will also be responsible for leading the onboarding and integration of multiple product teams onto the new Airflow platform
  • This role will also be responsible for creating Platform Run books and user guides as part of the product team onboard
  • This role will require significant hands-on work as you identify and implement custom Airflow operators with AWS
  • The Principal Software Engineer will be responsible for supporting and leading all Egen teams involved in the modernization project to ensure clarity in the roadmap and deliverables and that all technical blockers are removed
  • Finally, you will serve as an organizational leader at Egen, leading or designing workshops internally or for clients, or internal Egen teams, and supporting our hiring processes