Q

Qualified Health

Data-driven healthcare compliance and security

Senior Staff Engineer - Data Platform

Data EngineeringPosted on Nov 12
Full-timeRemote in USARemote

About the Job

Requirements

  • A bachelor's degree in Computer Science, Electrical Engineering, Mathematics, or a related field.
  • At least 12 years of software or data engineering experience, including a track record of designing and operating production data platforms at scale and setting architecture across multiple teams.
  • Experience working in a highly regulated domain such as healthcare, insurance, or finance.
  • Experience with the lakehouse stack, including Spark, Delta Lake, Unity Catalog, Workflows or equivalent, and performance and cost tuning at scale.
  • Experience building pipelines for machine learning or large language model applications, such as large language model-based extraction, embeddings and vector search, feature pipelines, or evaluation datasets.
  • Fluency in Python and SQL, with sound software engineering practices including testing, continuous integration and continuous delivery, infrastructure as code, and code review.
  • Demonstrated practical use of agentic coding tools such as Claude Code or similar in real delivery work, and sound judgment about when to trust, verify, and constrain them.
  • Experience owning ambiguous, cross-team problems end to end and influencing without direct authority.
  • Experience authoring design documents that others can build from.

Responsibilities

  • Set the technical direction for the Data Platform and build it hands-on.
  • Own the architecture of the lakehouse and the data pipelines, governance, and performance running on it, with Databricks at the center of the role.
  • Build and run AI pipelines alongside the data platform using Temporal for durable orchestration.
  • Set platform-wide strategy and architecture across multiple teams, anticipate problems a year or more ahead, shape the roadmap with leadership, and raise engineering effectiveness across the organization while building the hardest parts directly.
  • Architect and evolve the Databricks lakehouse, including medallion layering, Delta Lake, Unity Catalog governance, and workspace and compute strategy.
  • Design, build, and optimize Spark and SQL pipelines on Databricks, including incremental processing, data quality enforcement, performance tuning, and cost management.
  • Define data contracts, quality checks, access controls, and PHI-aware governance in partnership with security and compliance.
  • Build data foundations for AI products, including large language model-ready datasets, retrieval and vector-store pipelines, evaluation data, and interfaces that let AI and product teams safely consume governed data.
  • Design durable workflows on Temporal for AI pipelines, including scheduled and watermarked pulls from Databricks gold tables, file-ingestion workflows, and long-running large language model extraction and enrichment jobs.
  • Establish workflow patterns for retries, idempotency, versioning, and backfills, and keep the handoff between Databricks and Temporal clean, observable, and well governed.
  • Use AI coding agents daily and define the team's agentic development practices, including tooling, prompts, guardrails, code review practices, and test strategy for agent-written code.
  • Write and review technical designs, set platform-wide standards, and make build-versus-buy decisions that hold up over multiple years.
  • Mentor engineers across teams, unblock other pods, and communicate trade-offs to technical and non-technical stakeholders.

Desired Qualifications

  • Deep, hands-on expertise with Databricks is highly preferred; expertise in Snowflake or Microsoft Fabric is also aligned.
  • Experience with healthcare data such as claims, electronic health records, Fast Healthcare Interoperability Resources, or Health Level Seven, and HIPAA-compliant handling of protected health information.
  • Databricks certifications or experience with Delta Live Tables, Lakeflow, or Databricks Asset Bundles.
  • Experience with other orchestrators such as Airflow, Dagster, or Prefect, and a view on when each is the right tool.
  • Experience with cloud infrastructure such as Google Cloud Platform, Amazon Web Services, or Microsoft Azure, Terraform, and Kubernetes.
  • Contributions to open-source data, workflow, or AI infrastructure projects.

About the Company

Qualified Health

Data-driven healthcare compliance and security

Qualified Health AI provides data-driven software solutions for United States healthcare providers to manage patient data and meet regulatory requirements. The platform collects and processes healthcare information to streamline administrative tasks and support clinical decision-making, using automated tools while ensuring HIPAA compliance, privacy, and security. Beyond data processing, it plans collaborative features such as blogs and teaching games to facilitate knowledge sharing and professional development among healthcare professionals. The company differentiates itself by combining regulatory compliance services with community-building and educational tools, aiming to offer a single ecosystem where providers optimize daily operations, protect patient privacy, and foster continuous learning.

Company size
51-200
Company stage
Early VC
Total funding
$165.1M
Headquarters
Palo Alto, CA
Founded
2023

Recently Posted Jobs

Explore more opportunities at Qualified Health

See more jobs