Databricks Senior Architect

• Define and maintain theend-to-end architecturefor Primoris’ Databricks Lakehouse, including Medallion layer patterns, ingestion, transformation, serving, and AI/ML/GenAI workloads.

• Design and implementscalable, secure Databricks patterns(including serverless where appropriate) that power batch, streaming, ML,AI Gatewayand GenAI workloads, optimizing forperformance, cost, security, and reliability.

• Architect and implement solutions using the DatabricksMedallion architecture, Unity Catalog, and Delta Lake to support both analytical and operational use cases.

• Partner with Primoris’ enterprise / solution architecture, security, infrastructure, and application teams toalign Databricks with broader platform and integration standards.

Delivery & Operations

• Assist inarchitecting, developing, deploying, and migratingpriority use cases and workloads into production on the Databricks Lakehouse.

• Serve as thefirst point of contactfor technical challenges or questions related to development and production workloads (triage, root cause analysis, coordination of escalation as needed).

• Provide technical leadership and delivery oversight across hybrid teams, including internal employees, consulting partners, and offshore resources, ensuring seamless coordination, clear accountability, and consistent outcomes.

• Establish and championengineering standards(CI/CD, automated testing, observability, monitoring/alerting, incident runbooks) to ensureproduction-gradedelivery and operations.

• Design and reviewworkflows, jobs, and streaming pipelines(e.g., Auto Loader, DLT, structured streaming) for robustness, recoverability, and SLA adherence.

• Implement Databricksproduct innovations, private previews, and upgradesfollowing change management and release best practices, in coordination with Primoris IT and partners.

Data Engineering

• Build and reviewmulti-language notebooks(SQL, Python / PySpark) with a primary focus ondata engineeringanddata science / AI/MLworkflows.

• Design and guide implementation ofstreaming and batch data pipelines(Auto Loader, DLT, Structured Streaming) handling structured, semi-structured, and unstructured data.

Governance, Security & Compliance

• Implement and overseeRow-Level Security (RLS), column masking, data encryption/decryption, and other data protection controls using Unity Catalog and cloud-native capabilities.

• Architect and govern enterprise GenAI platforms on Databricks, including Unity Catalog, AI Gateway, Model Serving, and Vector Search, ensuring secure, compliant, observable, and cost-effective AI solutions across the organization.

• Work with security and compliance stakeholders to define and implementdata governance models, access patterns, data classification, auditing, and regulatory/contractual controls as required.

• Ensure all architectures and implementations align with Primoris’security, networking, and compliancestandards for Azure and Databricks.

Stakeholder Management & Communication

• Participate indiscovery workshopswith business and technical stakeholders to understand current-state architecture, data flows, pain points, and priority use cases.

• Translate findings intostrategic roadmaps, architecture options, and phased delivery plans with clear trade-offs, milestones, and ownership.

• Provideregular internal and external updateson progress, risks, and blockers to leadership, with clear options, mitigations, and recommendations.

Coaching, Standards & Reuse

• Define and evangelizepragmatic engineering and platform standards(coding patterns, data modeling, observability, cost optimization, workspace standards).

• Mentor engineers throughdesign and code reviews, knowledge-sharing sessions, and pairing on complex tasks.

• Develop and maintainreusable assets(reference architectures, templates, starter notebooks, Terraform/IaC patterns, runbooks) to accelerate adoption and improve consistency.

• Demonstrateproactive ownership: surface risks early, drive decisions, and remove blockers to maintain delivery momentum and stakeholder confidence.

Qualifications

• Bachelor’s degree inComputer Science, Information Systems, Engineering, Data Science, or equivalent practical experience.

• 8+ yearsof hands-on experience inData, Analytics, or AIroles (e.g., data engineering, data architecture, analytics platform engineering), with4+ yearsaccountable fortechnical delivery and architecture/tech-lead responsibilities.

• Proven experiencearchitecting and implementing Databricks solutionsusing theMedallion architecture, including ingestion, transformation, serving, and AI/ML workloads.

• Hands-on expertise implementing Databricks onMicrosoft Azure(required), including familiarity withAzure security, networking, identity, and storageconcepts.

• Experience designing and developinglarge-scale distributed systemsand big data solutions using technologies such asApache Spark™, Hadoop, or Cassandra.

• Proficiency inPython (including PySpark)andSQL, including performance tuning and best practices for large-scale data processing.

• Proficiency in designing and implementingAuto Loader,Delta Live Tables (DLT), andstreaming solutionsin Databricks to handle structured, semi-structured, and unstructured data.

• Experience withUnity Catalogand related capabilities (data governance, RBAC, lineage, auditing, and secure data sharing) or similar enterprise data governance tools.

• Demonstrated ability toattribute business value and outcomesto specific project deliverables and technical KPIs.

• Experienceconducting discovery workshops, creatingstrategic roadmaps, performingbusiness/process analysis, and managing or influencing the delivery ofcomplex data/AI programs or projects.

• Expertise implementing end-to-enddata security controlsin Databricks (RLS, column masking, encryption, key management), in alignment with enterprise security policies.

• Evidence ofcontinuous learningand staying current with the fast-evolving Databricks Lakehouse platform, Azure services, and modern data/AI patterns.

• Nice to have:

• One or moreDatabricks certifications(e.g.,Databricks Certified Data Engineer Professional,Databricks Certified Data Architect Associate/Professional).

• Deep hands-on knowledge ofBig Data Engineeringorcloud DWH technologies(e.g., Synapse, Snowflake, BigQuery, Redshift).

 Skills and Competencies

• Technical leadership: Ability to set architectural direction, make trade-off decisions, and guide engineers and partners toward production-ready solutions.

• Stakeholder management: Ability to manage expectations, influence decisions, and build trusted relationships with both technical and non-technical stakeholders (engineering, IT, security, operations, finance, business leadership).

• Communication: Excellent written and verbal communication skills; able to explain complex technical topics in clear, outcome-oriented language.

• Organization & execution: Exceptional organizational and time management skills; able to juggle multiple workstreams and drive them to completion.

• Attention to detail & quality: High standards for solution robustness, reliability, security, and documentation.

• Team building & mentoring: Strong leadership and team-building capabilities; experience mentoring engineers, establishing best practices, and raising the technical bar.

• Adaptability: Ability to adapt to changing priorities, work under pressure, and thrive in a fast-paced, evolving environment.

• Ownership mindset: Bias toward action, proactive identification of issues, and a strong sense of accountability for outcomes rather than just deliverables.