Z

Zillow

Online real estate platform with valuations

Principal Data Scientist

Data ScienceConfirmed live in the last 24 hours
Full-timeRemote in USA

About the Job

Requirements

  • At least 10 years of experience in Data Science, User Experience Research, or Applied Science, including meaningful experience measuring products whose customers are developers, internal teams, or platforms.
  • Experience leading measurement of platform products, including adoption, partner teams, and downstream outcomes.
  • Experience designing evaluations for complex user-facing systems using appropriate methods such as labeled test sets, human review, or LLM-as-a-judge.
  • Experience owning the design and analysis of A/B tests and causal-inference methodology in production, and selecting the appropriate method for difficult questions.
  • Deep experience designing primary metrics, key performance indicators, objective and key result metric definitions, and leading- and lagging-indicator systems from scratch.
  • Ability to distinguish measurement problems from instrumentation problems and address the correct one.
  • Ability to design event taxonomies, debug measurement methodology, and challenge schemas in collaboration with engineers.
  • Ability to write Python and SQL and to ship repeatable, reliable, documented instrumentation that other teams can adopt.
  • Ability to communicate rigorous measurement narratives to vice presidents and senior directors and surface relevant assumptions.
  • Ability to write code, model data, and prototype evaluations and dashboards as an individual contributor.
  • Preferred: PhD and 4+ years, MS and 6+ years, or BS and 8+ years of experience in Statistics, Econometrics, Data Science, User Experience Research, Machine Learning, Operations Research, or a related quantitative field.

Responsibilities

  • Define and operationalize the measurement strategy for the Connections Platform and broader Data & Infrastructure Platform team, from raw events through executive dashboards and causal-inference attribution.
  • Own evaluation of the Connections Platform’s LLM-based and agentic systems by designing evaluations, defining quality signals, and building frameworks, including LLM-as-a-judge and tools such as Braintrust.
  • Own the causal-inference and experimentation program: design and analyze A/B tests where possible, and develop quasi-experimental causal-inference designs such as switchbacks, difference-in-difference, synthetic control, and interrupted time-series when randomization is unavailable.
  • Advise the teams on what to measure and experiment on, challenge vanity metrics, identify missing leading indicators, and determine which signals reflect customer experience.
  • Connect adoption of platform agents and investments to business and revenue outcomes, beyond activity, speed, and quality signals.
  • Partner with engineering individual contributors to define and instrument events, schemas, and pipelines that produce trusted quality measurements.
  • Translate complex measurement realities and trade-offs into clear narratives for vice president and C-level partners.
  • Set the data-science measurement standard for platform teams, measuring platform adoption, quality, and impact with the rigor applied to consumer products.
  • Determine the impact of multidimensional product changes affecting consumers, partners, and developers, including business and revenue outcomes.

Desired Qualifications

  • Direct experience measuring complex, production user-facing systems, including nondeterministic conversational, LLM-based, or agentic products such as chat and voice assistants, lead or conversation funnels, and agent-driven workflows.
  • Fluency with modern evaluation methods, including LLM-as-a-judge and offline or online evaluation harnesses, or adjacent evaluation experience.
  • Experience measuring the impact of AI or agentic tooling on real outcomes, separating adoption and activity from productivity or quality gains, and reasoning about return on investment against spend.
  • Experience working in a Platform-as-a-Product model with internal teams as customers.
  • Fluency with quasi-experimental and causal-inference methods for situations where randomization is not possible, including difference-in-difference, synthetic control, interrupted time-series, and switchback or interleaved designs, with the ability to state assumptions and measurement limitations explicitly.

About the Company

Zillow

Online real estate platform with valuations

Zillow runs a real estate platform that helps people buy, sell, rent, and finance homes. It combines property listings with mortgage services and advertising, earning revenue from ads, leads for real estate agents, and mortgage origination. Its core product uses technology and the Zestimate to estimate home values and connect users with agents, lenders, and listings through an integrated online experience. Unlike services that focus on a single part of the market, Zillow offers a broad ecosystem that spans listings, valuations, and financing in one place, supported by data and network effects from a large user base. The company’s goal is to make moving to a new home easier by simplifying transactions and enabling users to complete more steps online, from search to financing to closing.

Company size
10,001+
Company stage
IPO
Headquarters
Seattle, WA
Founded
2005

Recently Posted Jobs

Explore more opportunities at Zillow

See more jobs