Data Engineer

<strong>Job Requirements </strong><br><p>At Quest Global, it’s not just what we do but how and why we do it that makes us different. With over 25 years as an engineering services provider, we believe in the power of doing things differently to make the impossible possible. Our people are driven by the desire to make the world a better place—to make a positive difference that contributes to a brighter future. We bring together technologies and industries, alongside the contributions of diverse individuals who are empowered by an intentional workplace culture, to solve problems better and faster.</p><p> </p><p><strong>Key Responsibilities</strong></p><ul><li>Develop, maintain, and optimize <strong>Spark-based data processing pipelines</strong> using <strong>Scala</strong>.</li><li>Work with distributed computing frameworks and resource management systems such as <strong>YARN</strong>.</li><li>Ingest, process, and manage large datasets using tools across the <strong>Hadoop ecosystem</strong> (HDFS, Hive, HBase, Oozie, etc.).</li><li>Write <strong>complex SQL queries</strong> for data extraction, transformation, validation, and performance optimization</li><li>Perform data validation, quality checks, and troubleshooting across datasets and jobs.</li><li>Monitor and improve data pipeline performance, ensuring high availability and reliability.</li><li>Participate in code reviews, documentation, and knowledge-sharing sessions.</li><li>Support ETL workflows, debugging production issues, and maintaining operational excellence</li></ul><p> </p><p>We are known for our extraordinary people who make the impossible possible every day. Questians are driven by hunger, humility, and aspiration. We believe that our company culture is the key to our ability to make a true difference in every industry we reach. Our teams regularly invest time and dedicated effort into internal culture work, ensuring that all voices are heard.</p><p> </p><p>We wholeheartedly believe in the diversity of thought that comes with fostering a culture rooted in respect, where everyone belongs, is valued, and feels inspired to share their ideas. We know embracing our unique differences makes us better, and that solving the worlds hardest engineering problems requires diverse ideas, perspectives, and backgrounds. We shine the brightest when we tap into the many dimensions that thrive across over 21,000 difference-makers in our workplace.</p><br><br><strong>Work Experience </strong><br><ul><li>Minimum <strong>2+ years</strong> of hands-on experience as a Data Engineer or in a similar role.</li><li>Strong programming skills in <strong>Python and Scala</strong>; exposure to Java is a plus.</li><li>Practical experience with <strong>Apache Spark (RDD/DataFrame APIs)</strong>.</li><li>Strong expertise in <strong>SQL querying</strong> (joins, subqueries, window functions, aggregations, performance tuning)</li><li>Good understanding of <strong>YARN</strong> for cluster resource management.</li><li>Working knowledge of <strong>Hadoop ecosystem components</strong>: </li><li><strong>HDFS, Hive, HBase, Oozie, Sqoop, MapReduce</strong></li><li>Experience with version control (Git) and CI/CD workflows.</li><li>Strong problem‑solving skills and ability to work in an agile environment.</li></ul><br><br>