A

Amazon

Sells products online and rents cloud computing

Senior Technical Program Manager - Robot Evaluation Operations

Program ManagementConfirmed live in the last 24 hours
Full-timeNew York, NY, USAIn-person

About the Job

Requirements

  • At least 5 years of technical product or program management experience.
  • At least 7 years of experience working directly with engineering teams.
  • Experience managing programs across cross-functional teams, building processes, and coordinating release schedules.
  • A bachelor's degree in Computer Science, Engineering, a related field, or equivalent experience.
  • Experience running test, validation, evaluation, or field operations programs for physical products such as robots, autonomous vehicles, or consumer devices.
  • Experience managing vendors, contractors, or operations teams that execute work on a schedule.

Responsibilities

  • Own the evaluation program from intake to results by taking requests from science and engineering teams, prioritizing them against shared robot and lab capacity, and publishing the campaign schedule.
  • Run evaluation campaigns in lab and home environments with clear entry and exit criteria and standard run procedures.
  • Build and maintain physical test environments, including home-like lab setups, object and fixture inventories, scene resets, and versioned configurations so results remain comparable across software releases and robot hardware revisions.
  • Establish in-home evaluation operations with legal, privacy, safety, and research partners, including scheduling, data handling, and on-site safety procedures.
  • Direct evaluation operators and vendor teams by managing training and certification, shift coverage, and scoring standards consistent across raters and sites.
  • Partner with hardware and fleet teams on robot availability for testing, including maintenance windows, repairs, and per-unit acceptance runs.
  • Enforce logging and traceability standards so every evaluation trial is reproducible and accounted for.
  • Deliver results through standard scorecards and dashboards, triage failures to the appropriate owning team, and provide evidence for launch and release go/no-go decisions.
  • Track throughput, valid-trial yield, cost per trial, and turnaround time; own the evaluation budget; and identify manual evaluations ready for automation or simulation.
  • Present evaluation status, risks, and tradeoffs to senior leadership, and communicate when demand exceeds capacity.

Desired Qualifications

  • Experience evaluating machine learning models or robot policies, or running data collection operations for robot learning.
  • Experience providing evidence for go/no-go or launch-readiness decisions.
  • Experience running tests in homes or with members of the public, including privacy and safety procedures.
  • Familiarity with experiment design and statistics, such as trial counts, confidence intervals, and scoring consistency across raters.
  • Ability to analyze data with SQL or Python and build dashboards.
  • Experience with simulation-based testing or hardware-in-the-loop test programs.

About the Company

Amazon

Sells products online and rents cloud computing

Amazon runs the largest online store in the United States and rents cloud computing to businesses and governments through Amazon Web Services. AWS pioneered renting computing power over the internet, and Amazon built its own delivery network to get orders to customers quickly. Shoppers buy from Amazon and from independent sellers on its store, and Prime members pay a subscription for fast free shipping, streaming video and music, and other perks. AWS rents out computing power, storage, databases, and AI tools so companies can run software without owning servers, and Amazon also makes Kindle e-readers and Echo speakers with the Alexa assistant. Jeff Bezos founded Amazon in 1994, and it is based in Seattle, Washington.

Company size
10,001+
Company stage
IPO
Headquarters
Seattle, WA
Founded
1994

Recently Posted Jobs

Explore more opportunities at Amazon

See more jobs