Data Engineer 3

Department Overview

Duties

Primary Responsibilities: 

• Application ownership and features 
• Design, build, test, and release new features across the web app's UI, APIs, and database, including KPI scorecards, peer benchmarking, multi-level performance views, goal and action tracking, mobile access, and multi-language support. 
• Help shape the app's architecture and technical design, run code reviews, and build reusable Vue 3 / Vuetify 3 components and services that keep Ops Standards apps consistent, scalable, and fast. 

Upgrades and modernization 

• Keep a roadmap for upgrades and technical debt. Upgrade frameworks and libraries, build tools, runtimes before they reach end of life, and database versions, and fix dependency and infrastructure vulnerabilities promptly using SonarQube, JFrog Xray, Snyk, Dependabot, and Wiz. 
• Extend automated unit and end-to-end test coverage (Playwright, pytest) and code quality gates in CI. 

Data engineering 

• Build and run batch pipelines that move visit, audit, and KPI data from the visit-capture application and the enterprise data warehouse into BigQuery. 
• Model the data into trusted, versioned datasets with dbt, schedule pipelines with Apache Airflow (Astronomer), and modernize remaining Talend jobs. 
• Build data quality checks (cleansing, standardization, de-duplication, mapping), agree Data Interface Contracts with the teams that provide and use the data, and send curated datasets back into the app, portals, and scorecards. 

Back end and integration 

• Build and maintain Python REST APIs and services (Flask / FastAPI) on Cloud Run, including the web API and machine-to-machine APIs, with Microsoft Entra ID (OAuth2 / OIDC with PKCE, JWT) authentication and role-based access control. 
• Own the app database (Cloud SQL for MySQL), its schema, and its data loads, including SQL Server staging and Oracle source dependencies. Write efficient SQL and add caching where it helps. 
• Onboard and maintain APIs on the Gravitee gateway, and work with network and security teams on Akamai, DNS, Cloud Load Balancing, and Cloud Armor. 

Cloud, DevOps, and observability 

• Maintain GitHub Actions CI/CD pipelines, Terraform, and the Dev, Test, and Prod environments on GCP, including single-page app releases to Cloud Storage and API services on Cloud Run. 
• Set up monitoring, logging, and alerting with Cloud Monitoring, Cloud Logging, Datadog, and PagerDuty, and track uptime, performance, and cost. 

Production support 

• Act as the technical owner for incidents, including application defects, wrong data shown in the app, pipeline and integration failures, and performance problems. Run root cause analysis, put permanent fixes in place, and keep runbooks and architecture documents up to date. 
• Support the yearly market setup (KPIs, comparison groups, visit requirements), onboarding of new markets, and release management for global users. 

Collaboration and ways of working 

• Work with product owners, architects, data engineers, security teams, and global and market stakeholders using Agile delivery, mentor engineers, and help set engineering standards. 
• Use approved AI coding assistants (such as GitHub Copilot and Gemini Code Assist) across the lifecycle for code, tests, SQL and dbt models, debugging, and documentation, and review and validate every AI-generated change. 
•  

Qualifications

• 7–11 years of professional experience in full-stack software development. 
• Data engineering (minimum 3 years, hands-on): production pipelines on GCP using BigQuery, Apache Airflow (Astronomer), and dbt, with data modeling and data quality practices. 
• Front end: TypeScript / JavaScript, HTML5, and CSS / SASS, with hands-on experience building single-page apps in Vue 3 with Vuetify 3 and Vite, charting and data grid libraries such as Highcharts and AG Grid, internationalization (vue-i18n), and accessibility and front-end performance. 
• Back end: Python REST frameworks (Flask, FastAPI), microservices, and REST API design, including versioning and authentication with OAuth2 / OIDC (PKCE, JWT) on Microsoft Entra ID. 
• Databases: production-quality SQL with MySQL (Cloud SQL), SQL Server, and Oracle, and query tuning. 
• GCP: Cloud Run, Cloud Storage, Cloud SQL, BigQuery, and IAM. 
• DevOps and quality: Git and GitHub Actions, Docker, and Terraform; Playwright end-to-end testing; and security and quality tools such as SonarQube, JFrog, and Snyk. 
• A track record of upgrading frameworks, patching vulnerabilities, and reducing technical debt in production apps. 
• Hands-on experience using AI coding assistants and LLM tools to improve engineering productivity, with sound judgment on code quality and security, and responsible use that keeps confidential and personal data out of prompts. 
• Background in Computer Science, Engineering, or a related field with a bachelor's degree or higher. 
• Preferred: audit, inspection, or compliance data with versioned questionnaires and scoring; quick service restaurant or food safety experience; Gravitee API gateway and Akamai CDN; Cloud Load Balancing, Cloud Armor, and Pub/Sub; Talend; Node.js; React; Jest / Vitest unit testing; and Google Professional Data Engineer or Professional Cloud Developer certification. 
• Good to know: Java / Spring Boot or .NET; change data capture (Datastream, Debezium); semantic engineering (semantic layers and governed metric definitions, for example dbt Semantic Layer, and ontology design) and knowledge graphs (Neo4j or Spanner Graph, with Cypher, GQL, or SPARQL).