Q

Quest Global

Engineering services, digital transformation, asset management

Data Engineering Specialist

Data EngineeringUpdated on Nov 7
Full-timePune, Maharashtra, IndiaNo salary listedIn-person

About the Job

Requirements

  • The role requires 8+ years of relevant experience in data engineering, big data platforms, and production pipeline delivery.
  • The candidate must have hands-on experience designing data pipelines using NiFi, Spark/PySpark, Hive SQL, and Python.
  • The candidate must understand Hadoop concepts, HDFS storage patterns, data partitioning, and distributed processing.
  • The candidate must have experience managing Airflow DAGs, dependencies, retries, monitoring, and failure handling.
  • The candidate must be able to identify pipeline bottlenecks and perform performance tuning across ingestion, transformation, and orchestration layers.
  • The candidate must be able to support critical production workloads and solve related problems.

Responsibilities

  • Design and develop robust data ingestion and ETL pipelines for batch and workflow-based processing requirements.
  • Build, optimize, and maintain data processing solutions using Apache Spark, PySpark, and Apache NiFi.
  • Develop, schedule, monitor, and manage workflows in Apache Airflow to ensure reliable orchestration.
  • Work on the Cloudera Data Engineering platform and the broader Hadoop ecosystem, including HDFS and Hive.
  • Write efficient Hive SQL and Python code for data transformation, validation, and operational automation.
  • Support, troubleshoot, and optimize production data pipelines for performance, reliability, and scalability.
  • Collaborate with data architects, analysts, platform teams, and business stakeholders to understand requirements and deliver data solutions.
  • Follow engineering best practices for code quality, version control, documentation, deployment, and monitoring.

About the Company

Quest Global

Engineering services, digital transformation, asset management

QuEST Global provides engineering and lifecycle services across aerospace, automotive, energy, healthcare, and transportation, focusing on digital transformation and asset management. It uses edge computing, cloud, AI/ML, and industrial IoT to deliver tailored engineering solutions through long-term partnerships, service contracts, project-based engagements, and strategic collaborations. It differentiates itself with a broad industry footprint, global delivery model, and emphasis on partnerships, talent development, and workplace culture. Its goal is to help clients improve operational efficiency and drive business growth by transforming engineering processes and assets with digital tools and data-driven management.

Company size
10,001+
Company stage
Growth Equity (Venture Capital)
Total funding
$581M
Headquarters
Greenville, SC
Founded
1997

Recently Posted Jobs

Explore more opportunities at Quest Global

See more jobs