Z

ZettaSquared

Senior Databricks Platform Architect

Cloud EngineeringPosted on Nov 11
Full-timeHouston, TX, USA

About the Job

Requirements

  • A bachelor's degree in Computer Science, Engineering, Information Systems, or a related field is required.
  • At least 7 years of experience in data architecture, cloud architecture, or enterprise platform engineering is required.
  • At least 5 years of hands-on Databricks implementation experience is required.
  • Proven experience deploying and configuring new Databricks environments from inception through production is required.
  • Strong expertise in the Databricks Lakehouse Platform, Unity Catalog, Delta Lake, Databricks Workflows, MLflow, Apache Spark, PySpark, SQL, and Python is required.
  • Experience implementing Infrastructure as Code using Terraform is required.
  • A deep understanding of enterprise security, networking, governance, and compliance requirements is required.
  • Experience designing cloud-native architectures in Azure, AWS, or GCP is required.
  • Strong stakeholder management and leadership experience is required.
  • Applicants must be currently authorized to work in the United States on a full-time basis.

Responsibilities

  • Lead the installation, configuration, and deployment of Databricks environments.
  • Design and configure Databricks workspaces, clusters, cluster policies, compute resources, and networking.
  • Deploy and configure Unity Catalog, Delta Lake, MLflow, and Databricks Workflows.
  • Implement Infrastructure as Code, with Terraform preferred, for platform provisioning and lifecycle management.
  • Establish CI/CD pipelines supporting code promotion and environment management.
  • Design and implement enterprise security architecture for Databricks.
  • Configure identity management, role-based access control, single sign-on, SCIM provisioning, and integration with enterprise identity providers.
  • Implement data governance frameworks, metadata management, lineage, and cataloging.
  • Establish data protection controls, encryption, auditing, monitoring, and compliance standards.
  • Define governance processes using Unity Catalog and enterprise data management tools.
  • Establish patterns and standards for data ingestion, transformation, orchestration, and analytics.
  • Enable AI/ML development through MLflow and Databricks AI capabilities.
  • Define best practices for structured, semi-structured, streaming, and batch data processing.
  • Support enterprise modernization and migration initiatives.
  • Act as the technical lead for Databricks platform implementation projects.
  • Guide engineers, administrators, and architects throughout deployment and adoption.
  • Facilitate architecture reviews and technical decision-making.
  • Communicate project status, risks, and recommendations to executive leadership.
  • Work onsite near the 290/Beltway 8 area in Houston, Texas, for the project and from an office when not engaged with a client.

Desired Qualifications

  • Databricks Certified Professional or Databricks Certified Architect certification is preferred.
  • Azure Solutions Architect, AWS Solutions Architect, or Google Professional Cloud Architect certification is preferred.
  • Experience implementing enterprise data governance frameworks is preferred.
  • Experience migrating legacy data warehouses and analytics platforms to Databricks is preferred.
  • Knowledge of DevOps, DataOps, and MLOps practices is preferred.
  • Experience supporting AI/ML and Generative AI workloads is preferred.

About the Company

Company size
Not listed
Company stage
Not listed
Total funding
Not listed
Headquarters
Not listed
Founded
Not listed

Recently Posted Jobs

Explore more opportunities at ZettaSquared

See more jobs