N

NorthBay Solutions

AWS migrations, data analytics, ML services

Senior Data Engineer

Data EngineeringConfirmed live in the last 24 hours
Full-timeAbu Dhabi - United Arab EmiratesNo salary listed

About the Job

Requirements

  • A bachelor's degree in Computer Science, Information Technology, Software Engineering, or a related field.
  • Six to ten years of professional experience in Data Engineering, Data Warehousing, or Enterprise Data Platform development.
  • Hands-on experience with Microsoft Azure Data Services and cloud-based data engineering solutions.
  • Practical expertise in Azure Data Factory, Azure Synapse Analytics, Azure Data Lake Storage Gen2, and Azure SQL Database.
  • Experience designing, developing, and maintaining large-scale ETL/ELT pipelines and data integration frameworks.
  • Proficiency in SQL, query optimization, and database performance tuning.
  • Hands-on experience with PySpark, Apache Spark, and distributed data processing frameworks.
  • Strong programming skills in Python; exposure to Scala or Java is an advantage.
  • Understanding of dimensional modeling, data warehousing principles, and modern lakehouse architectures.
  • Experience processing structured, semi-structured, and unstructured data.
  • Knowledge of data governance, data quality, metadata management, and data security best practices.
  • Experience with REST APIs, enterprise data integration patterns, and connectivity with business applications.
  • Familiarity with Git, CI/CD pipelines, and DevOps practices.
  • Analytical, troubleshooting, and problem-solving skills.
  • Communication and collaboration skills to work across technical and business teams.

Responsibilities

  • Design, develop, and maintain scalable data platforms and data pipelines using Microsoft Azure services.
  • Build and optimize batch and real-time data ingestion frameworks for structured, semi-structured, and unstructured data sources.
  • Design and implement data lake, data warehouse, and lakehouse architectures to support analytics and reporting requirements.
  • Develop, manage, and optimize ETL/ELT pipelines using modern cloud-native data engineering practices.
  • Implement data transformation, cleansing, validation, and data quality frameworks to ensure data accuracy and consistency.
  • Collaborate with business stakeholders, data analysts, data scientists, and application teams to translate requirements into scalable data solutions.
  • Optimize data storage, processing, and query performance across enterprise data platforms.
  • Implement data security, governance, monitoring, metadata management, and compliance best practices across Azure environments.
  • Integrate enterprise data platforms with AI/ML solutions, business intelligence tools, and enterprise applications.
  • Develop and maintain data models, reusable components, and efficient data processing workflows.
  • Participate in architecture reviews, code reviews, technical troubleshooting, and continuous improvement initiatives.
  • Support data platform reliability, scalability, high availability, and operational excellence.
  • Implement version control, CI/CD pipelines, and DevOps practices for data engineering workloads.
  • Contribute to technical documentation, knowledge sharing, and mentoring of team members where required.

Desired Qualifications

  • Experience with Microsoft Fabric, OneLake, Dataflows, and Fabric Data Engineering workloads.
  • Hands-on experience with Databricks, Delta Lake, and lakehouse implementations.
  • Knowledge of real-time streaming technologies such as Azure Event Hubs, Apache Kafka, or Azure Stream Analytics.
  • Experience supporting AI/ML and advanced analytics workloads through enterprise data platforms.
  • Familiarity with Power BI datasets, semantic models, and enterprise reporting architectures.
  • Experience with data governance tools such as Microsoft Purview.
  • Microsoft Azure Data Engineering certifications.
  • Experience working in Agile/Scrum environments.
  • Exposure to DataOps and MLOps practices.
  • Knowledge of containerization technologies such as Docker and Kubernetes.
  • Experience with Infrastructure as Code tools, including Terraform, ARM templates, or Bicep.
  • Familiarity with Snowflake, Google BigQuery, or other cloud data warehouse platforms.
  • Experience with enterprise-scale data migration, modernization, and cloud transformation projects.
  • Understanding of Generative AI, vector databases, and data platforms supporting AI workloads.

About the Company

NorthBay Solutions

AWS migrations, data analytics, ML services

NorthBay Solutions is an AWS Premier Partner that helps large organizations move their databases and applications to Amazon Web Services, modernize software, and build data analytics, DevOps/DataOps pipelines, and ML/AI solutions. Its services cover database and application migrations, data and analytics workloads, DevOps and DataOps practices, and technology modernization across cloud and data platforms. The company operates by delivering engineering and consulting projects in an agile, quality-focused way to accelerate time-to-market and maximize value from cloud investments. What sets NorthBay apart is its status as an AWS Premier Partner with deep expertise in cloud migrations, data engineering, and AI/ML workloads, along with a track record of working with recognizable brands. The goal is to help clients securely migrate to AWS, modernize their applications, and unlock data-driven insights and intelligent capabilities quickly and reliably.

Company size
201-500
Company stage
Not listed
Total funding
Not listed
Headquarters
Cambridge, MA
Founded
2007

Recently Posted Jobs

Explore more opportunities at NorthBay Solutions

See more jobs