Data Engineer

Job Description:

Our AI & Digital Innovations teams bring the latest GenAI techniques plus a deep understanding of industry dynamics and corporate functions to help clients create the most value from data. Involved in specific engagements involving the building of new, bespoke advanced AI solutions for clients and internally. Leveraging your knowledge of algorithms, data structures, and performance optimism and experience with processing and interpreting data sets. Responsible for developing and translating computer algorithms into prototype code and maintaining, organizing, and identifying trends in large data sets. Proficiency in SQL database design and creating process documentation. Familiarity with the computer coding languages python, Azure Synapse to oversee real-time business metric aggregation, data warehousing and querying, schema and data management, and related duties. Develop technical solutions to improve access to data and data usage. Understand data needs and advise company on technological resources. Collaborating closely with other data scientists, machine learning engineers, and stakeholders to understand the project’s objectives and translate these into innovative, data-driven solutions.

Skills:   

Azure Data Engineer - We are looking for a Mid-level Azure Data Engineer with a minimum of 5 years of experience working in Data science projects, preferably GenAI projects. 

An ideal candidate will have following qualifications:

• Hands on experience with Azure Data Factory, Azure Databricks and Fabric for data integration and real-time analytics.
• Design, develop, and implement a high-performance Python coding with a focus on efficiency, scalability, and performance.
• Create a robust and reusable code using Python that can independently generate creative content and directly contribute to the project’s goals
• Familiarity with data modeling, warehousing, and ETL development processes.
• Expert level on using standard ML libraries such as Numpy and Pandas
• Advanced level on using ML frameworks PyTorch/TensorFlow on Azure cloud ecosystem.
• Collaborate closely with data scientists, machine learning engineers, and other stakeholders to understand their requirements and translate these into data-driven solutions
• Troubleshoot, debug, and resolve any issues that occur within the generative AI system development, ensuring that the models perform optimally.
• Background on Unit, Integration, and System testing.  Conduct tests and experiments to enhance the performance of overall system and document the results for further improvements.
• Conduct code reviews, assist other developers, and train team members, as required
• Ensure that developed system comply with industry standards and best practices, and that they meet the project’s requirements.
• Document all processes, specifications, and training procedures, and maintain stringent version control to ensure the high quality, accuracy, and replicability of generative models.
• Work independently and a team player.
• Be creative and dynamic in advancing newer ideas.
• Azure certification would be a plus

Education: A university degree is preferred