EXL

Analyzes business data and runs outsourced operations

Data Engineer

Data EngineeringConfirmed live in the last 24 hours

Full-timeGurugram, Haryana, IndiaNo salary listedIn-person

About the Job

Requirements

At least 8 years of experience in data engineering or ETL roles.
At least 4 years of experience in Azure cloud ETL leadership.
Azure certifications, such as Azure Analytics Specialty or Solutions Architect.
Experience with Azure Data Lake Storage, Blob Storage, Azure SQL Database, and Synapse Analytics.
Experience connecting to APIs, on-premises databases, and flat files in CSV, Parquet, or JSON formats.
Experience with Azure Data Factory for orchestration and Databricks connectors.
Strong knowledge of Apache Spark, PySpark, and Spark SQL for distributed processing.
Experience with data cleaning and normalization, including handling nulls, duplicates, and schema evolution.
Experience with performance optimization techniques, including partitioning, caching, and broadcast joins.
Experience implementing Delta Lake ACID transactions, time travel, and schema enforcement.
Experience building Azure Data Factory pipelines to orchestrate Databricks notebooks.
Experience using Azure Key Vault for secure credential management.
Experience with Azure Monitor and logging for ETL job monitoring and alerting.
Knowledge of networking and security, including VNET integration and private endpoints.
Strong experience in data engineering and problem-solving skills.

Responsibilities

Design and build data systems and pipelines.
Develop data processing workflows, optimize data storage, and ensure data accuracy.
Collaborate with data scientists and analysts to meet data requirements and resolve data issues.
Design and implement data pipelines.
Optimize data processing and storage.
Ensure data solutions meet performance standards.
Provide technical support.
Collaborate with stakeholders.

Data Engineers design and build data systems and pipelines. Responsibilities include developing data processing workflows, optimizing data storage, and ensuring data accuracy. You will collaborate with data scientists and analysts to meet data requirements and resolve data issues. Strong experience in data engineering and problem-solving skills are required.

8+ years in data engineering/ETL roles, with at least 4+ years in Azure cloud ETL leadership. Azure certifications (e.g., Azure Analytics Specialty, Solutions

Required Qualifications:

Azure Data Sources: Azure Data Lake Storage (ADLS), Blob Storage, Azure SQL Database, Synapse Analytics. External Sources: APIs, on-prem databases, flat files (CSV, Parquet, JSON).
Tools: Azure Data Factory (ADF) for orchestration, Databricks connectors.
Apache Spark: Strong knowledge of Spark (PySpark, Spark SQL) for distributed processing.
Data Cleaning & Normalization: Handling nulls, duplicates, schema evolution.
Performance Optimization: Partitioning, caching, broadcast joins.
Delta Lake: Implementing ACID transactions, time travel, and schema enforcement.
Azure Data Factory (ADF): Building pipelines to orchestrate Databricks notebooks.
Azure Key Vault: Secure credential management.
Azure Monitor & Logging: For ETL job monitoring and alerting.
Networking & Security: VNET integration, private endpoints.

Design and implement data pipelines, Optimize data processing and storage, Ensure data solutions meet performance standards, Provide technical support, Collaborate with stakeholders.

EXL

Data Engineer

About the Job

Requirements

Responsibilities

About the Company

EXL

Recently Posted Jobs