A

Amazon

Sells products online and rents cloud computing

Software Development Engineer - Multiple Teams

Applied Machine LearningConfirmed live in the last 24 hours
Full-timeCupertino, CA, USAIn-person

About the Job

Requirements

  • A bachelor's degree in computer science or equivalent is required.
  • At least 3 years of non-internship professional software development experience is required.
  • At least 3 years of non-internship experience designing or architecting new and existing systems, including design patterns, reliability, and scaling, is required.
  • Fundamental knowledge of machine learning and large language models, their architectures, training and inference lifecycles, and work experience optimizing model execution is required.
  • Software development experience in C++ or Python is required; experience in at least one language is required.
  • A strong understanding of system performance, memory management, and parallel computing principles is required.
  • A deep understanding of computer architecture and operating-systems-level software, and working knowledge of parallel computing, is required.
  • Proficiency in debugging, profiling, and applying software engineering best practices in large-scale systems is required.

Responsibilities

  • Help lead efforts to build distributed inference support for PyTorch in the Neuron SDK.
  • Tune models to maximize performance and efficiency on customer AWS Trainium and Inferentia silicon and servers.
  • Design, develop, and optimize machine learning models and frameworks for deployment on custom machine learning hardware accelerators.
  • Participate in all stages of the machine learning system development lifecycle, including distributed-computing-based architecture design, implementation, performance profiling, hardware-specific optimizations, testing, and production deployment.
  • Build infrastructure to systematically analyze and onboard multiple models with diverse architectures.
  • Design and implement high-performance kernels and features for machine learning operations, leveraging the Neuron architecture and programming models.
  • Analyze and optimize system-level performance across multiple generations of Neuron hardware.
  • Conduct detailed performance analysis using profiling tools to identify and resolve bottlenecks.
  • Implement optimizations such as fusion, sharding, tiling, and scheduling.
  • Conduct comprehensive testing, including unit and end-to-end model testing, with continuous deployment and releases through pipelines.
  • Work directly with customers to enable and optimize their machine learning models on AWS accelerators.
  • Collaborate across compiler, runtime, framework, and hardware teams to optimize machine learning workloads and develop innovative optimization techniques.
  • Debug performance issues, optimize memory usage, shape the Neuron inference stack, create metrics, implement automation and other improvements, and resolve the root cause of software defects.
  • Deliver solutions to customers, participate in design discussions and code reviews, and communicate with internal and external stakeholders.
  • Work cross-functionally and provide technical input to help drive business decisions.

Desired Qualifications

  • A master's degree or Ph.D. in computer science or equivalent is preferred.
  • Familiarity with PyTorch, just-in-time compilation, and ahead-of-time tracing is preferred.
  • Familiarity with CUDA kernels or equivalent machine learning or low-level kernels is preferred.
  • Experience developing performant kernels such as CUTLASS or FlashInfer is preferred.
  • Familiarity with syntax and tile-level semantics similar to Triton is preferred.
  • Experience with online or offline inference serving using vLLM, SGLang, TensorRT, or similar platforms in production environments is preferred.

About the Company

Amazon

Sells products online and rents cloud computing

Amazon runs the largest online store in the United States and rents cloud computing to businesses and governments through Amazon Web Services. AWS pioneered renting computing power over the internet, and Amazon built its own delivery network to get orders to customers quickly. Shoppers buy from Amazon and from independent sellers on its store, and Prime members pay a subscription for fast free shipping, streaming video and music, and other perks. AWS rents out computing power, storage, databases, and AI tools so companies can run software without owning servers, and Amazon also makes Kindle e-readers and Echo speakers with the Alexa assistant. Jeff Bezos founded Amazon in 1994, and it is based in Seattle, Washington.

Company size
10,001+
Company stage
IPO
Headquarters
Seattle, WA
Founded
1994

Recently Posted Jobs

Explore more opportunities at Amazon

See more jobs