Data Engineer 3

Department Overview

Duties

• Build and operate edge-to-cloud ingestion for restaurant events and equipment telemetry using Confluent Kafka (with NATS JetStream and MQTT at the edge), Cloud Run consumers, and Dataflow or Pub/Sub where appropriate. 
• Handle the realities of device data, including late and out-of-order events, duplicates, intermittent connectivity, and schema changes, with idempotent and replayable pipelines. 
• Model time-series and event data in BigQuery, and maintain the operational Cloud SQL for PostgreSQL stores behind shift-level APIs. 
• Maintain equipment, device, and event reference data, aligned to restaurant location and hierarchy master data, so that signals roll up consistently. 
• Produce feature-ready datasets for computer vision, equipment troubleshooting and quality, and AI agent use cases in partnership with data science and AI teams. 
• Serve data and APIs to shift-level operations apps and restaurant performance applications owned by partner application teams. 
• Manage event schemas and data contracts in Confluent Schema Registry (JSON Schema / CloudEvents, Avro, Protobuf) and Data Interface Contracts with IoT and application teams. 
• Develop orchestration with Apache Airflow (Astronomer) and transformations with dbt, using Git, GitHub Actions, and automated testing. 
• Implement data quality, freshness, and latency checks with alerting (OpenTelemetry, New Relic, Datadog, PagerDuty), and optimize streaming and query costs. 
• Support new market onboarding for restaurant devices such as order accuracy scales, and act as level-2 support for production pipeline incidents. 
• Use approved AI coding assistants (such as GitHub Copilot and Gemini Code Assist) across the lifecycle for code, tests, SQL and dbt models, debugging, and documentation, and review and validate every AI-generated change. 
• Document pipelines and runbooks, mentor junior engineers, and coordinate with teams across time zones as needed. 

Qualifications

• 7–11 years of professional experience, with 5+ years of hands-on data engineering. 
• Advanced Python and production-grade SQL (window functions, analytical transforms, query plan tuning). 
• Hands-on experience with event streaming on Confluent Kafka, including Schema Registry and JSON Schema / CloudEvents event contracts, with Google Cloud Pub/Sub as a plus. 
• Strong GCP data experience: BigQuery, Cloud Storage, Dataflow / Dataproc, Cloud Run, Cloud SQL for PostgreSQL, Firestore, and IAM. 
• Hands-on experience with Apache Airflow (Astronomer) and dbt. 
• Experience modeling time-series or event data at scale. 
• Practical experience with Docker, Git, GitHub Actions CI/CD, and Terraform. 
• Expert knowledge of data quality functions such as cleansing, standardization, de-duplication, mapping, and hierarchy management. 
• Hands-on experience using AI coding assistants and LLM tools to improve engineering productivity, with sound judgment on code quality and security, and responsible use that keeps confidential and personal data out of prompts. 
• Background in Computer Science, Engineering, or a related field with a bachelor's degree or higher. 
• Preferred: Java for streaming and data processing (Beam Java SDK) and integration with Java services; IoT and edge messaging (NATS JetStream, MQTT, OpenTelemetry); restaurant equipment data such as drive-thru timers, order accuracy scales, and computer vision; supporting BI users in Tableau or other visualization tools; ML feature pipelines (Vertex AI); Google Professional Data Engineer certification; and exposure to quick service restaurant or retail operations. 
• Good to know: Scala; semantic engineering (semantic layers and governed metric definitions, for example dbt Semantic Layer, and ontology design) and knowledge graphs (Neo4j or Spanner Graph, with Cypher, GQL, or SPARQL).