Compute System Architecture Engineer - AI Infra

We are building reusable, reliable AI infrastructure for engineering teams, transforming technical documents, source code, and engineering data into accessible knowledge and enabling AI agents to perform complex engineering tasks. In this role, you will design and develop knowledge platforms, LLM service integrations, and agent execution frameworks, taking AI capabilities from prototypes to maintainable production systems for knowledge retrieval, test development, code review, and failure analysis.

What you’ll be doing:

• Design and develop reusable AI platform components and services that provide consistent interfaces for model access, knowledge retrieval, tool calling, and workflow execution.
• Build data ingestion and processing pipelines for heterogeneous sources, including documents, code, and tables, supporting parsing, structured extraction, incremental updates, versioning, and provenance.
• Develop retrieval-augmented generation (RAG) and knowledge infrastructure, combining vector search, keyword search, reranking, and structured knowledge to improve retrieval quality and context relevance.
• Develop agentic and multi-agent workflows with tool integration, context and state management, task scheduling, failure recovery, and human review.
• Build evaluation and observability systems using datasets, automated tests, regression benchmarks, and execution traces to measure and improve correctness, reliability, latency, and cost.
• Own platform deployment and maintenance, including access controls, data isolation, recovery mechanisms, and release processes, supporting reliable use across multiple users and applications.
• Collaborate with domain experts and application developers to translate engineering needs into platform capabilities and continuously improve them through user feedback.

What we need to see:

• Bachelor’s degree in Computer Science, Software Engineering, Electrical Engineering, or a related field, or equivalent experience. An MS or PhD is a plus.
• Hands on experience in software engineering, infrastructure, data platforms, machine learning engineering, or related areas.
• Strong Python programming skills and software engineering fundamentals, with familiarity with Linux, version control, automated testing, and CI/CD.
• Hands-on development and deployment experience in at least one of the following: LLM application infrastructure, RAG systems, agent platforms, or model-serving systems.
• Understanding of the core mechanisms and limitations of LLM applications, including model APIs, embeddings, context management, structured outputs, tool calling, and evaluation.
• Experience with backend services, databases, and data processing, including concurrency, caching, task state management, failure recovery, and API design.
• Strong debugging, testing, and analytical skills, with the ability to diagnose issues through logs, metrics, and experiments and independently validate AI-generated code and results.
• Good communication, collaboration, and ownership, with the ability to learn new domains and drive solutions from requirements through delivery and maintenance.

Ways to stand out from the crowd:

• Experience with enterprise knowledge bases, complex document parsing, knowledge graphs, ontology modeling, or data quality management.
• Experience developing or integrating agent frameworks, multi-agent systems, MCP, or other tool protocols, with the ability to understand and extend their underlying mechanisms.
• Experience with AI evaluation, security testing, permission isolation, prompt-injection defenses, or reliability engineering for nondeterministic systems.
• Experience optimizing model deployment, inference services, batching, caching, or resource scheduling, with an understanding of quality, throughput, latency, and cost tradeoffs.
• Demonstrated ownership of production AI systems or meaningful contributions to relevant open-source projects and C++ development experience, or a background in computer architecture, GPUs, compilers, hardware verification, or engineering developer tools.

NVIDIA is widely considered to be one of the technology world’s most desirable employers. We have some of the most forward-thinking and hardworking people in the world working for us. If you’re creative, autonomous, and love a challenge, we want to hear from you