Staff Data Engineer

The Government Technology Agency (GovTech) is the lead agency driving Singapore’s Smart Nation initiatives and public sector digital transformation. As the Centre of Excellence for Infocomm Technology and Smart Systems (ICT & SS), GovTech develops the Singapore Government’s capabilities in Data Science & Artificial Intelligence, Application Development, Smart City Technology, Digital Infrastructure, and Cybersecurity. 
 
At GovTech, we offer you a purposeful career to make lives better. We empower our people to master their craft through continuous and robust learning and development opportunities all year round. Our GovTechies embody our Agile, Bold and Collaborative values to deliver impactful solutions. 
 
GovTech aims to transform the delivery of Government digital services by taking an "outside-in" view, putting citizens and businesses at the heart of everything we do. Play a part in Singapore’s vision to build a Smart Nation and embark on your meaningful journey to build tech for public good. 
 
Join us to advance our mission and shape your future with us today! 
 
Learn more about GovTech at tech.gov.sg

About the role

The National Environment Agency (NEA) is the leading public organisation responsible for ensuring a clean and sustainable environment for Singapore. Meteorological Service Singapore (MSS), part of NEA, provides meteorological and climate information for Singapore. We are modernising the systems that ingest, manage, and make this data available for forecasting, climate analysis, and downstream services.

We are seeking a Staff Data Engineer to lead the technical design, delivery, and evolution of MSS’s Weather Data Lake.

You will help establish a governed, trusted foundation for meteorological and climate data. Working closely with MSS domain specialists and engineering teams, you will translate complex requirements into reliable data pipelines, clear architectural decisions, and maintainable platform capabilities.

This is a hands-on technical leadership role spanning data engineering, software engineering, and cloud infrastructure. You will set technical direction, lead critical implementation work, and strengthen engineering practices across the team. The platform runs on Microsoft Azure through Government on Commercial Cloud (GCC 2.0), using technologies including Databricks, Delta Lake, Unity Catalog, Apache Spark, and Terraform.

[What you will be working on]

Lead platform architecture and technical direction

• Set the technical direction for MSS’s Weather Data Lake, balancing data processing, storage, security, governance, reliability, and cost.
• Work with domain specialists and engineering teams to translate meteorological and climate data requirements into technical designs and delivery plans.
• Define clear boundaries and interfaces between ingestion systems, the data lake, and downstream services, ensuring critical operational workflows do not depend directly on the availability of the analytical platform.
• Document architectural decisions and build alignment on technical priorities, dependencies, and trade-offs.

Build reliable data pipelines and quality controls

• Design, build, and optimise batch and streaming pipelines for meteorological observations, operational data, and international data exchanges.
• Establish consistent approaches to data validation, schema evolution, incremental processing, reprocessing, and traceability.
• Develop data quality approaches for operational forecasting and historical climate analysis, with appropriate controls for timeliness, validation, and the handling and propagation of corrections.
• Lead historical data migration, including validation, reconciliation, cutover planning, and integration with downstream consumers.

Establish secure and maintainable platform foundations

• Manage Databricks workspaces, storage, networking, and access controls through modular, version-controlled Terraform.
• Establish CI/CD and environment-management practices for repeatable deployments, clear separation between staging and production, and controlled changes.
• Implement data governance using Unity Catalog and supporting platform controls, including access policies, classification metadata, audit logging, and data lineage, in line with applicable IM8 and NEA requirements.
• Evaluate platform capabilities against user needs, security requirements, operational readiness, and long-term cost and maintainability.

Grow engineering capability

• Remain hands-on in complex engineering work while guiding technical design and implementation across the team.
• Mentor engineers through design reviews, code reviews, pairing, and collaborative problem-solving.
• Establish shared practices and documentation that make the platform easier to build, operate, and extend.
• Help engineers develop the technical judgement and context to own platform capabilities independently.

[What we are looking for]

• Substantial experience in data engineering, platform engineering, or backend software engineering, typically gained through 8 or more years of relevant work.
• A track record of leading complex technical initiatives, combining architectural ownership with hands-on delivery and collaboration across engineers or teams.
• Strong hands-on experience building and operating production workloads on Databricks, including Delta Lake, Unity Catalog, and Spark-based data processing.
• Proficiency in Python and SQL, with experience developing and optimising large-scale batch or streaming pipelines using Spark.
• Experience building and operating Azure-based data platforms, including Azure Data Lake Storage Gen2, Blob Storage, identity and access management, and private networking.
• Strong Terraform experience, with sound practices for modular infrastructure, version control, and CI/CD.
• Experience implementing data quality controls, governance, and lineage in production systems.
• The ability to turn ambiguous requirements into clear technical direction, document decisions, and explain trade-offs to technical and non-technical stakeholders.
• A track record of developing other engineers through mentoring, constructive feedback, and shared engineering practices.
• Strong communication and collaboration skills, with the ability to work closely with domain specialists to understand unfamiliar data and workflows.

[Nice to have] 

• Experience with GCC 2.0 or other regulated cloud environments with strict security and change-management requirements.
• Experience with change data capture, incremental processing, and reliable propagation of data updates.
• Experience migrating legacy data systems, including historical data validation, reconciliation, and cutover planning.
• Familiarity with Azure API Management and integrating data platforms with downstream APIs and services.
• Experience with scientific, environmental, or time-series datasets where data quality, provenance, and historical corrections matter.
• Familiarity with World Meteorological Organization (WMO) standards and data exchanges, including WIS 2.0.
• Experience using AI-assisted development tools, with sound practices for reviewing generated code, testing changes, and protecting sensitive information.
  
  

This role will start off on a 2 year direct contract

What we offer you:

GovTech is an equal opportunity employer committed to fostering an inclusive workplace that values diverse voices and perspectives, as we believe that diversity is the foundation to innovation.   

Our employee benefits are based on a total rewards approach, offering a holistic and market-competitive suite of perks. These include leave benefits to meet your work-life needs and employee wellness programs. 

We champion flexible work arrangements (subject to your job role) and trust that you will manage your own time to deliver your best, wherever you are, and whatever works best for you.  

Learn more about life inside GovTech at go.gov.sg/GovTechCareers. 
Stay connected with us on social media at go.gov.sg/ConnectWithGovTech

Careers@Gov Salary Range: 0.0 - 0.0