E

Equinix

Global data center, colocation, interconnection provider

Principal Product Manager - Agent Intelligence

Technical Product ManagementPosted on Nov 4
Full-timeHybrid

About the Job

Requirements

  • At least 7 years across product management and engineering, including at least 3 years as a product manager on technical platform, infrastructure, data, or machine-learning products.
  • A systems engineering background, with several years building or operating distributed systems, data platforms, infrastructure, or machine-learning systems as an engineer, or equivalent hands-on depth.
  • A track record of building and running evaluation, quality, or observability systems for production machine-learning or large language model products at scale, with demonstrated results.
  • Direct experience with multi-component systems such as agents, tool use, retrieval pipelines, or orchestrated services, including finding which component caused a failure.
  • Strong evaluation judgment, including knowing when to use code-based checks, large-language-model judges, or human review; calibrating a judge against experts; and recognizing when metrics do not reflect quality.
  • Hands-on data skills, including writing SQL, reading traces and logs, and prototyping an evaluation or analysis in Python or similar.
  • Ability to lead without authority across engineering, data science, and business teams, and to explain trade-offs clearly to senior leaders.

Responsibilities

  • Define quality for each agent and the end-to-end system across correctness, grounding in sources, completeness, tool-use accuracy, workflow adherence, latency, cost, and business outcomes.
  • Turn product requirements and acceptance criteria into measurable evaluations with clear thresholds and owners.
  • Set scoring standards and rubrics that apply consistently across agents, workflows, regions, and industries for both experts and automated judges.
  • Own the roadmap for offline and online evaluation, including golden datasets, scenario and regression suites, LLM-as-judge pipelines calibrated against human experts, and human-in-the-loop review.
  • Integrate evaluation into agent releases through evaluation-based quality gates in CI/CD, preventing model, prompt, tool, or data changes from reaching users without passing agreed thresholds.
  • Measure reliability using repeated trials, consistency across runs, confidence intervals, and confidence calibration, in addition to accuracy.
  • Evaluate individual components and complete systems, including retrieval, entity resolution, ranking, tool and MCP calls, agent handoffs, and final outputs.
  • Build trajectory- and trace-level evaluation that traces end-to-end failures to the responsible component and tests how errors spread across handoffs.
  • Define component contracts, including schemas, provenance, and confidence, so quality can be checked at every boundary.
  • Partner with engineering on tracing and telemetry across agents, tools, and data sources so outputs can be traced to their inputs, sources, and decisions.
  • Build monitoring and alerts for drift in inputs, data sources, models, and tool behavior to catch regressions before users do.
  • Provide leaders and product teams with a trusted view of quality, reliability, cost, and adoption by agent, workflow, and segment.
  • Analyze production traces, user edits, dismissals, ratings, and expert audits to identify recurring failure modes, edge cases, and data gaps.
  • Turn each confirmed failure into a regression test, product requirement, or training and tuning data, and track it through to a fix.
  • Run experiments and A/B tests to measure whether changes improve accuracy, reliability, adoption, and business outcomes.
  • Build evaluations and controls that enforce data boundaries, keep internal data internal, restrict customer-facing content to approved material, and prevent confidential details from crossing between accounts.
  • Partner with security, legal, and data governance teams to meet regulatory and confidentiality requirements, including for customers in regulated industries.
  • Set evaluation standards adopted by other product and engineering teams, and influence teams the role does not manage.
  • Write strategy, specifications, and executive briefs that align engineering, data science, sales, and leadership.
  • Stand up a shared evaluation platform and golden datasets for core agent workflows during the first year.
  • Put evaluation gates in front of every agent release.
  • Ship end-to-end tracing across multi-agent workflows so failures are traced to the responsible component.
  • Ensure production failures become regression tests within days.

Desired Qualifications

  • Experience with large-language-model evaluation and observability tools, tracing standards such as OpenTelemetry, and agent protocols such as MCP and A2A.
  • Experience with retrieval, ranking, recommendation, confidence calibration, or learning-from-feedback systems.
  • Experience with experimentation platforms and causal measurement.
  • Experience with enterprise data such as CRM, billing, product usage, or network and infrastructure data.
  • Experience with regulated customers or data governance, including financial services, privacy, or confidentiality controls.
  • A degree in computer science, engineering, or another quantitative field, or equivalent experience.

About the Company

Equinix

Global data center, colocation, interconnection provider

Equinix provides data center space, colocation, and interconnection services to businesses worldwide. Customers rent space to house servers and use interconnected networks inside Equinix data centers, enabling low-latency data transfer between networks and cloud providers. Unlike competitors that focus only on space, Equinix combines a global footprint with a large ecosystem of over 10,000 customers for direct, high-speed connections. Its goal is to enable fast, reliable digital operations and support customers’ digital transformation and multi-cloud strategies.

Company size
10,001+
Company stage
IPO
Headquarters
Redwood City, CA
Founded
1998

Recently Posted Jobs

Explore more opportunities at Equinix

See more jobs