Full-Time

Lead Data Engineer

Multiple Teams

Posted on 9/8/2026

Egen

Egen

1,001-5,000 employees

Cloud-based AI platform development and consulting

No salary listed

Hyderabad, Telangana, India

Hybrid

Bachelor's

Category
Data & Analytics (1)
Required Skills
Microsoft Azure
FastAPI
Python
Airflow
Git
BigQuery
Apache Spark
Apache Kafka
Postgres
ETL
Data Engineering
AWS
Redis
DevOps
Oracle
Databricks
Google Cloud Platform

Get referred to Egen

See people who can refer or advise you

Requirements
  • 10+ years of hands-on experience in Python for backend or data engineering projects.
  • Strong understanding and working experience with Google Cloud Platform services, especially Dataflow, BigQuery, Cloud Functions, and Cloud Composer.
  • Working experience with Azure Data Factory, Azure Databricks, and Azure Data Lake Storage Gen2.
  • Solid understanding of data pipeline architecture, data integration, and transformation techniques.
  • Experience with version control systems such as GitHub and knowledge of continuous integration and continuous delivery practices.
  • Experience with Apache Spark, Kafka, Redis, FastAPI, Airflow, and Google Cloud Composer DAGs.
  • Strong experience in Structured Query Language with at least one enterprise database such as SQL Server, Oracle, or PostgreSQL.
  • Experience with PySpark.
  • Experience in migrating data from on-premises data sources to cloud platforms.
  • Bachelor's degree in Computer Science, a related field, or equivalent experience.
Responsibilities
  • Gather requirements and conduct discovery activities with customers.
  • Design, architect, develop, test, and maintain robust and scalable extract, transform, and load and extract, load, and transform data pipelines.
  • Architect enterprise solutions using Google Cloud Platform, Azure, Databricks, PySpark, and Spark SQL.
  • Use Google Cloud Platform services including Dataflow for real-time and batch data processing, Cloud Functions for serverless computing, BigQuery for data warehousing and analytics, Cloud Composer for data workflow orchestration, Google Cloud Storage for large-scale data management, Identity and Access Management for access control and security, and Cloud Run for containerized applications.
  • Develop production-grade Databricks notebooks and workflows.
  • Build data transformation pipelines using PySpark and Spark SQL.
  • Implement Delta Lake architecture and Bronze, Silver, and Gold data layers using the Medallion Architecture.
  • Implement Databricks workflows and jobs with dependency management.
  • Tune Spark jobs for large-scale data processing and optimize cluster configuration and compute utilization.
  • Implement partitioning, caching, and file-size optimization strategies.
  • Ingest data from various sources and apply transformation and cleansing logic to ensure high-quality data delivery.
  • Implement and enforce data quality checks, validation rules, and monitoring.
  • Collaborate with data scientists, analysts, and engineering teams to understand data needs and deliver data solutions.
  • Manage version control using GitHub and participate in continuous integration and continuous delivery pipeline deployments for data projects.
  • Write complex Structured Query Language queries for data extraction and validation from relational databases such as SQL Server, Oracle, and PostgreSQL.
  • Document pipeline designs, data flow diagrams, and operational support procedures.
  • Provide post-delivery support.
Desired Qualifications
  • Experience with Amazon Web Services.

Egen.ai builds data-driven solutions by combining cloud computing, AI, and platform development to help organizations optimize operations, reduce costs, and improve decision-making. It delivers tailored consulting and scalable platforms, partnering with cloud and data platform providers across industries, including government and enterprise clients. Its solutions collect and analyze data from clients’ systems, deploy AI-powered features, and run on cloud infrastructure to support predictive maintenance, mobile ordering, and other use cases. The goal is to accelerate clients' data and platform value chains, delivering measurable cost savings and operational improvements.

Company Size

1,001-5,000

Company Stage

Pre-seed

Total Funding

$1.3M

Headquarters

Naperville, Illinois

Founded

2000

Get referred to Egen

See people who can refer or advise you

Simplify Jobs

Simplify's Take

What believers are saying

  • Qarik acquisition broadened Egen's enterprise reach across communications, healthcare, life sciences, and retail.
  • Christy Mayr's 2026-07-07 appointment targets the biggest AI bottleneck: workforce adoption.
  • Egen's 700-plus specialists across Canada, Europe, India, and the U.S. support global delivery.

What critics are saying

  • Egen depends heavily on Google Cloud, exposing it to partner-policy changes and pricing pressure.
  • Services margins face commoditization as clients internalize AI implementation after Gemini Enterprise deployments.
  • A Google Cloud ecosystem slowdown or partner consolidation threatens Egen's core revenue base by 2027.

What makes Egen unique

  • Egen acquired Qarik on 2024-10-24, creating 800-plus Google Cloud specialists.
  • Egen is a Google Cloud Premier Partner and Gemini Enterprise preferred partner in 2026.
  • Egen launched Agentic Change Enablement on 2026-07-07, combining AI deployment with change management.

Help us improve and share your feedback! Did you find this helpful?

Benefits

Fun team outings

Health insurance

401(K)

Dental & vision insurance

Paid time-off

Food & drinks

Parental leave

Company News

IT Digest
Jul 8th, 2026
Egen launches Agentic Change Enablement Practice to drive enterprise AI adoption.

Egen launches Agentic Change Enablement Practice to drive enterprise AI adoption. Egen, a leading cloud, data, and platform engineering firm, announced the launch of its dedicated Agentic Change Enablement Practice. The specialized service offering is engineered to bridge the execution gap between initial technology implementation and long-term business value, helping enterprises optimize employee workflows and organizational structures for autonomous AI integration. As multi-national corporations accelerate capital allocations toward generative and agentic artificial intelligence, an operational bottleneck has shifted to center stage. While modern engineering groups can deploy autonomous software engines relatively quickly, the ultimate yield on that investment hinges on human alignment. Without targeted operational restructuring, extensive workforce upskilling, clear performance metrics, and rigorous governance structures, complex AI investments can stall out in the pilot phase. "Agentic AI will not create value simply because it is deployed," said Alim Somani, CEO of Egen. "Value comes when agents are embedded into real workflows, people understand how their work is changing, and organizations have the governance, training, and measurement needed to scale adoption. Our Agentic Change Enablement Practice brings those pieces together so enterprises can move from experimentation to durable enterprise impact." Redesigning Workflow architecture via Google Cloud integration. Egen, operating as a Google Cloud Premier Partner, will leverage its expanded organizational footprint to guide companies through the entire lifecycle of agentic software transformation. The newly formed practice unifies technical agent engineering and underlying platform data modeling with specialized corporate change management. The strategic framework targets three distinct transformation phases to maximize technology ROI: * Workflow Orchestration and Mapping: Auditing core corporate business units to identify, isolate, and prioritize the exact processes best suited for autonomous agent execution. * Workforce Evolution and Training: Designing structured learning curricula, transition roadmaps, and alignment communication channels to prepare front-line personnel for collaborative human-to-AI operating environments. * Performance Measurement at Scale: Establishing real-time auditing dashboards and objective telemetry tracking matrices to continuously evaluate how active AI systems impact overall business velocity. The practice integrates natively with Google Cloud's modern technology landscape - including Gemini Enterprise toolsets - to construct automated agents grounded in verified corporate context. This system ensures that all automated workflows operate inside a secure, compliant ecosystem featuring enterprise-grade sovereignty controls. Appointing digital transformation veteran Christy Mayr to lead practice. To accelerate market penetration and build out the division's consulting methodologies, Egen has appointed digital transformation leader Christy Mayr as Head of Agentic Change Enablement. Mayr and her dedicated team of organizational design and technology implementation specialists will concentrate on accelerating the adoption velocity of newly deployed digital systems across multi-tier enterprise models. By structuring strategic learning tracks, enhancing cross-departmental decision matrices, and managing stakeholder alignment across complex operations, the practice enables corporate leadership to convert experimental AI ideas into predictable, compliant, and measurable financial outcomes. The service layer is active and available for onboarding immediately, providing a comprehensive operational framework for organizations seeking to scale their underlying digital capacities.

PR Newswire
Jul 7th, 2026
Egen launches Agentic Change Enablement Practice to accelerate enterprise AI adoption

Egen has launched its Agentic Change Enablement Practice to help organisations move from AI pilots to scaled enterprise adoption. The consulting firm, a Google Cloud Premier Partner with over 700 technology specialists, will guide companies through workflow redesign, team adoption of AI-enabled processes, and impact measurement. The practice aims to address the gap between AI technology deployment and creating real business value. It focuses on embedding AI agents into workflows, workforce readiness, and governance frameworks using Google Cloud technologies including Gemini Enterprise. Egen appointed digital transformation veteran Christy Mayr to lead the new capability. The firm works across Canada, Europe, India, and the United States, specialising in cloud, data, platforms, and AI implementation.

PR Newswire
Oct 24th, 2024
Egen Announces Acquisition of Qarik, Strengthens Position as Leading Cloud, Platform and AI Services Provider in the Google Cloud Ecosystem

/PRNewswire/ -- Egen, a technology services company with leading capabilities across Google Cloud infrastructure, product and data engineering, and artificial...