A

Amazon

Sells products online and rents cloud computing

Senior ML Software Engineer - Data Plane

Applied Machine LearningConfirmed live in the last 24 hours
Full-timeCupertino, CA, USAIn-person

About the Job

Requirements

  • A bachelor's degree in computer science or equivalent is required.
  • At least 7 years of full software development life cycle experience, including coding standards, code reviews, source control management, build processes, testing, and operations experience.
  • Strong proficiency in C and C++.
  • Strong Linux systems knowledge.
  • Experience developing compute kernels for GPUs, DSPs, or custom accelerators.
  • A proven track record of owning and delivering complex software features end-to-end.

Responsibilities

  • Develop and optimize compute kernels for a custom ML accelerator architecture, targeting production-level performance for large language model inference.
  • Implement and validate large language model architectures end-to-end, from PyTorch model definition through distributed execution on custom hardware.
  • Integrate custom accelerator backends into open-source ML serving frameworks, including vLLM and PyTorch, with scheduler extensions, memory management, and model parallelism.
  • Build and maintain test infrastructure for model correctness validation across CPU, GPU, simulator, and hardware targets.
  • Profile and optimize inference workloads by identifying bottlenecks, instrumenting critical paths, and improving latency and throughput from simulation through hardware bringup.
  • Own features end-to-end, from design through implementation, testing, and integration into the broader software stack.
  • Contribute to continuous integration and continuous delivery pipelines that gate model and kernel changes on correctness and performance regressions.
  • Mentor engineers, drive design reviews, and raise the engineering bar across the team.

Desired Qualifications

  • Knowledge of machine learning and large language model fundamentals, including transformer architecture, training and inference lifecycles, and optimization techniques.
  • Knowledge of machine learning frameworks including JAX, PyTorch, vLLM, SGLang, Dynamo, TorchXLA, and TensorRT.
  • Familiarity with speculative decoding, key-value cache optimization, or other large language model serving optimizations.
  • Experience with distributed systems, including collective communication, remote direct memory access, or high-speed interconnect programming.
  • Experience with hardware simulation environments and model validation workflows.
  • Experience using large language models or code-generation agents as part of daily development workflow.

About the Company

Amazon

Sells products online and rents cloud computing

Amazon runs the largest online store in the United States and rents cloud computing to businesses and governments through Amazon Web Services. AWS pioneered renting computing power over the internet, and Amazon built its own delivery network to get orders to customers quickly. Shoppers buy from Amazon and from independent sellers on its store, and Prime members pay a subscription for fast free shipping, streaming video and music, and other perks. AWS rents out computing power, storage, databases, and AI tools so companies can run software without owning servers, and Amazon also makes Kindle e-readers and Echo speakers with the Alexa assistant. Jeff Bezos founded Amazon in 1994, and it is based in Seattle, Washington.

Company size
10,001+
Company stage
IPO
Headquarters
Seattle, WA
Founded
1994

Recently Posted Jobs

Explore more opportunities at Amazon

See more jobs