Senior Data Engineer-Databricks & Data Modeling

Job Description

Position Summary

The Senior Data Engineer - Databricks & Data Modeling will develop, deploy, and maintain production data pipelines and reusable data models within Winsupply's Databricks lakehouse on Google Cloud. This hands-on role combines strong Databricks engineering skills with practical relational and dimensional modeling expertise to convert complex source data into trusted Silver- and Gold-layer data products. Working within the Data Architecture & Engineering team, this individual will implement approved architectural standards, collaborate with business stakeholders, Data Governance, and IT, and take ownership of reliable, efficient delivery from source ingestion through production support.

Accountabilities/Responsibilities

• Build, test, deploy, and support production ingestion and transformation pipelines using Databricks, Delta Lake, Unity Catalog, Apache Spark, Python/PySpark, and Databricks SQL.
• Independently design and implement relational and dimensional data models for assigned data products, including fact tables, conformed dimensions, star schemas, and slowly changing dimensions (SCD Type 1/2).
• Translate core business processes and relationships among Customer, Product, Vendor, Purchasing, Inventory, Sales, and Pricing entities into accurate Silver-layer structures and reusable Gold-layer data products, working with Data Architecture leadership on complex model decisions.
• Ingest and integrate batch and streaming data from ERP, relational, legacy/mainframe, fixed-width or delimited files, APIs, and cloud storage; handle schema changes, incremental loads, and CDC as appropriate.
• Implement business keys, table grain, joins, deduplication, history handling, and source-to-target mappings that produce repeatable, traceable results.
• Create and automate data quality checks, reconciliation, pipeline monitoring, alerting, exception handling, and recovery procedures for production operations.
• Tune Spark workloads, Databricks SQL queries, Delta table layouts, job configuration, and compute usage to improve performance, reliability, and cost efficiency.
• Develop reusable, maintainable code with Git, peer reviews, unit/integration tests, CI/CD, and repeatable Databricks deployment practices.
• Partner with business process owners and Data Governance to understand operational workflows, validate entity definitions and business rules, and use that context to inform sound data modeling, quality, lineage, and access decisions.
• Document pipelines, models, transformation logic, dependencies, and runbooks so solutions can be maintained and supported by the broader team.
• Troubleshoot data defects and pipeline failures, identify root causes, and drive fixes through production with appropriate communication and ownership.
• Collaborate with the architecture team, DaaS, IT, and delivery partners; contribute practical recommendations and mentor peers while supporting approved technical direction.

Competencies for Success

• Has delivered and supported Databricks pipelines and data products in production from raw ingestion through trusted analytical outputs.
•  Demonstrates strong modeling judgment grounded in business process knowledge: understands core entities, grain, keys, relationships, SCD Type 1/2, fact/dimension design, and reporting impacts.
•  Troubleshoots distributed data processing, data-quality discrepancies, and performance issues with a structured, evidence-based approach.
•  Communicates clearly, challenges proposed solutions constructively, and collaborates across technical and business teams.
•  Takes ownership of code quality, documentation, reliability, cost awareness, and delivery commitments. Qualifications/Experience

Qualifications

Minimum Qualifications

• 7+ years of related professional experience in data engineering, data warehousing, analytics engineering, or comparable roles.
•  3+ years of hands-on Databricks production experience with Delta Lake, Unity Catalog, Spark/PySpark, and Databricks SQL.
•  3+ years of meaningful relational or dimensional data modeling experience, including personally designing and deploying fact/dimension or integrated business models.
•  Foundational understanding of core business processes and entities (e.g., Customer, Product, Vendor, Purchasing, Inventory, Sales, and Pricing), with the ability to validate relationships with business stakeholders and apply that context to data modeling and governance decisions.
•  Experience building production ETL/ELT pipelines, implementing incremental processing, and supporting operational data quality and monitoring.
• Strong SQL and Python/PySpark skills, with hands-on Git, testing, code review, and CI/CD practices.
•  Associate degree in Computer Science, Information Systems, or a related field; equivalent professional experience or relevant technical certifications may be considered.

Preferred Qualifications

• Bachelor's degree in Computer Science, Information Systems, Data Engineering, or a related discipline.
• Databricks Certified Data Engineer Professional or comparable Databricks certification.
• Experience with Databricks on Google Cloud, Lakeflow pipelines/jobs, Auto Loader, CDC, and Declarative Automation Bundles.
• Kimball dimensional modeling, enterprise data governance, semantic modeling for Looker, or advanced Spark performance optimization.
• Experience integrating IBM/mainframe, DB2, distribution/supply-chain, financial, or pricing data.

Additional Information

Physical Demands

The physical demands here are representative of those that must be met to successfully perform the essential job functions with or without reasonable accommodations:

• Sitting for extended periods of time
• Dexterity of hands and fingers to operate a computer keyboard, mouse, power tools, and other computer components
• Lifting and transporting of moderately heavy objects, such as computers and peripherals
• Occasional travel to interact with local companies, regional offices and/or support implementation of solutions

We are committed to inclusion, diversity, and equal employment opportunity, regardless of race, color, ancestry, religion, sex, gender, national origin, sexual orientation, age, citizenship, marital status, disability, veteran status, or any other factor protected by applicable federal, state, or local laws.