A

Amazon

Sells products online and rents cloud computing

Software Engineer 2 - AI/ML, Neuron Inference

Backend EngineeringConfirmed live in the last 24 hours
Full-timeCupertino, CA, USAIn-person

About the Job

Requirements

  • Have 3+ years of non-internship professional software development experience.
  • Have a bachelor's degree or foreign equivalent in Computer Science, Engineering, Mathematics, or a related field.
  • Have knowledge of Python and/or C++ programming.
  • Have 3+ years of non-internship design or architecture experience with design patterns, reliability, and scaling of new and existing systems.
  • Have 1+ years of experience designing and developing large-scale, multi-tiered, multi-threaded, embedded, or distributed software applications, tools, systems, and services using C#, C++, Java, or Perl.
  • Have experience debugging, profiling, and implementing software engineering best practices in large-scale systems.
  • Have a strong understanding of system performance, memory management, and parallel computing principles.

Responsibilities

  • Design, develop, and optimize machine learning models and frameworks for deployment on custom machine learning hardware accelerators.
  • Participate in all stages of the machine learning system development lifecycle, including distributed-computing-based architecture design, implementation, performance profiling, hardware-specific optimizations, testing, and production deployment.
  • Build infrastructure to systematically analyze and onboard multiple models with diverse architectures.
  • Understand and produce Neuron Kernel Interface kernels, and design and implement high-performance kernels and features for machine learning operations using the Neuron architecture and programming models.
  • Analyze and optimize system-level performance across multiple generations of Neuron hardware, using knowledge of the underlying Trainium hardware architecture to guide optimization decisions.
  • Conduct detailed performance analysis using profiling tools to identify and resolve bottlenecks.
  • Implement optimizations such as fusion, sharding, tiling, and scheduling.
  • Conduct comprehensive testing, including unit and end-to-end model testing, with continuous deployment and releases through pipelines.
  • Work directly with customers to enable and optimize their machine learning models on AWS accelerators.
  • Collaborate across teams to develop optimization techniques, and share proposals, findings, and learnings with internal developers and external customers through designs and documentation.
  • Build distributed inference support for PyTorch in the Neuron SDK and tune models to maximize performance and efficiency on AWS Trainium.
  • Collaborate with compiler, runtime, framework, and hardware teams to optimize machine learning workloads.
  • Debug performance issues, optimize memory usage, and contribute to the Neuron inference stack across Amazon and the open-source community.
  • Create metrics, implement automation and other software architecture improvements, and resolve the root causes of software defects.
  • Deliver solutions to customers, participate in design discussions and code reviews, and communicate with internal and external stakeholders.
  • Contribute technical input to cross-functional business decisions.
  • Review operational metrics and address alerts from owned systems.
  • Design features, write and review code, debug issues with teammates, participate in design discussions and sprint planning, and document systems.

Desired Qualifications

  • Have experience with machine learning and large language model fundamentals.
  • Have knowledge of machine learning model architecture and inference.
  • Have experience with performant machine learning kernel development, such as CUDA, Triton, or FlashInfer.
  • Have knowledge of syntax and tile-level semantics similar to Triton.
  • Have experience with vLLM, SGLang, TensorRT, or similar platforms in production environments.
  • Have experience with PyTorch, just-in-time compilation, and ahead-of-time tracing.

About the Company

Amazon

Sells products online and rents cloud computing

Amazon runs the largest online store in the United States and rents cloud computing to businesses and governments through Amazon Web Services. AWS pioneered renting computing power over the internet, and Amazon built its own delivery network to get orders to customers quickly. Shoppers buy from Amazon and from independent sellers on its store, and Prime members pay a subscription for fast free shipping, streaming video and music, and other perks. AWS rents out computing power, storage, databases, and AI tools so companies can run software without owning servers, and Amazon also makes Kindle e-readers and Echo speakers with the Alexa assistant. Jeff Bezos founded Amazon in 1994, and it is based in Seattle, Washington.

Company size
10,001+
Company stage
IPO
Headquarters
Seattle, WA
Founded
1994

Recently Posted Jobs

Explore more opportunities at Amazon

See more jobs