AI Infrastructure Architect Manager

About Accenture

Accenture is a leading global professional services company, providing a broad range of services and solutions in strategy, consulting, digital, technology and operations. Combining unmatched experience and specialized skills across more than 40 industries and all business functions - underpinned by the world’s largest delivery network - Accenture works at the intersection of business and technology to help clients improve their performance and create sustainable value for their stakeholders. With 750K people serving clients in more than 120 countries, Accenture drives innovation to improve the way the world works and lives. Visit us at www.accenture.com

About Global Network

Accenture Strategy shapes our clients’ future, combining deep business insight with the understanding of how technology will impact industry and business models. Our focus on issues such as digital disruption, redefining competitiveness, operating and business models as well as the workforce of the future helps our clients find future value and growth in a digital world. Today, every business is a digital business. Digital is changing the way organisations engage with their employees, business partners, customers, and communities - how they manufacture and deliver products and services, and how they run their organisations. This is our unique differentiator. We seek people who recognise and understand the impact that digital and technology have on every industry and every sector, and share our passion to shape unique strategies that allow our clients to succeed in this environment.

To bring this global perspective to our clients, Accenture Strategy’s services include those provided by our Global Network - a distributed management consulting organisation that provides management consulting and strategy expertise across the client lifecycle. Approximately 10,000 consultants are part of this rapidly expanding network, providing specialised and strategic industry and functional consulting expertise from key locations around the world. Our Global Network teams complement our in-country teams to deliver cutting-edge expertise and measurable value to clients all around the world. For more information visit www.accenture.com/capabilitynetwork

Global Network Videos

Video Title

External Link

Accenture Global Network

https://www.youtube.com/watch?v=-92pvOH1d_k

Accenture in One Word

https://www.youtube.com/watch?v=t1Fo8uNWZ-0

MBA Careers: What makes Accenture different

https://www.youtube.com/watch?v=5bg4u5Sczm8

Accenture Inclusion & The Power of Diversity

https://www.youtube.com/watch?v=2g88Ju6nkcg

AI Infrastructure Architect - Manager

Practice Overview

Skill / Operating Group

Technology Consulting - AI Infrastructure Advisory

Level

Manager

Location

Gurugram / Mumbai / Bangalore / Pune / Kolkata

Travel

Expected travel could be anywhere between 0–100%

Why Technology Consulting

The Technology Consulting business within Global Network invents the future for clients by providing them with the right guidance, design thinking and innovative solutions for technological transformation. Specialise in AI infrastructure consulting to transform the world’s leading organisations by: Helping Clients rethink their AI compute, networking, retrieval, and agent orchestration layers to scale enterprise AI reliably. Enhancing your Skillset with hyperscaler AI infrastructure (AWS, Azure, GCP), Kubernetes, Terraform, MLOps/LLMOps tooling, and agentic AI architecture. Transforming Businesses by designing next-generation AI infrastructure foundations that move clients from AI pilots to enterprise-scale production deployments.

Principal Duties & Responsibilities

Overview

Design and deliver end-to-end AI infrastructure architecture for enterprise clients - setting the architecture vision for compute, networking, retrieval, agent orchestration, and the tooling layers that underpin every AI and agentic workload.

Serve as the trusted technical advisor to client architects and technology leads on AI infrastructure decisions, translating architectural choices into clear trade-offs across cost, performance, scalability, and security.

This is a design and architecture role, not a hands-on build role. The mandate is to define standards, reference architectures, and governance frameworks that engineering teams implement.

This role focuses on AI infrastructure, platform engineering, governance, and architecture. It does not focus on model development, prompt engineering, model fine-tuning, or AI application development. Applications from AI Engineers, Prompt Engineers, or ML practitioners without infrastructure architecture experience are unlikely to be a strong fit.

Key Responsibilities

1. AI Compute & Infrastructure Architecture

• Design cloud AI infrastructure for enterprise workloads - GPU/CPU compute, Kubernetes (AKS, EKS, GKE), managed ML compute, networking, identity, and security across AWS, Azure, or GCP.
• Define reference architecture for AI Center of Excellence infrastructure - model hosting, agent environments, and the compute and networking beneath them.
• Architect compute strategy - GPU SKU selection, GPU economics and capacity planning, provisioned vs pay-as-you-go inference, and shared vs dedicated capacity models.
• Design AI-specific landing zones with network isolation, private connectivity for AI services, and AI guardrails at the workload boundary.

2. AI Platform Architecture

• Design enterprise AI platform architectures using Azure AI Foundry, AWS Bedrock, and GCP Vertex AI.
• Define model catalog governance, model lifecycle management, platform operating models, and AI self-service enablement patterns.
• Architect integration patterns between Azure AI Foundry, AWS Bedrock, and GCP Vertex AI and the underlying Kubernetes (AKS / EKS / GKE) compute and networking infrastructure.

3. Inference Layer Architecture

• Architect model serving strategy across managed cloud endpoints and self-hosted inference (e.g. vLLM/Triton on GPU-backed Kubernetes) - right pattern per use case.
• Design inference routing - model fallback chains, multi-model routing, and load balancing across provisioned and on-demand deployments.
• Define latency and throughput architecture for real-time and batch inference, including token-level latency budgets and streaming response design.
• Architect inference-layer caching - prompt caching and semantic caching to reduce redundant model calls and control token economics and cost.

4. Retrieval / RAG Architecture

• Design vector database architecture for RAG - select and size across cloud AI search, vector-enabled databases, and dedicated vector stores based on scale and latency needs.
• Architect embedding pipeline infrastructure - embedding model selection, reindexing strategy, and compute/storage patterns for chunking at enterprise scale.
• Define hybrid search architecture (vector, keyword, metadata filtering) and the infrastructure to support it at scale.
• Architect multi-tenant vector store isolation - ensuring one business unit’s embedded data cannot leak into another’s retrieval results through index- and access-level boundaries.
• Design knowledge graph and GraphRAG infrastructure - graph database hosting and its integration pattern with the vector retrieval layer.

5. Agentic & Tool Calling Architecture

• Architect tool calling infrastructure - how agents securely discover, authenticate to, and invoke internal APIs, MCP servers, and third-party connectors.
• Design MCP server hosting and governance - versioning, security, and reuse across agents and business units.
• Define the integration security boundary - scoped credentials, rate limiting per agent-to-tool call, and audit logging of every tool invocation.
• Architect agent runtime environments for agent orchestration frameworks (LangGraph, Semantic Kernel, CrewAI) on Kubernetes or serverless containers.
• Design multi-agent orchestration infrastructure, including agent-to-agent communication patterns and short-term/long-term agent memory storage.

6. Identity, Access & AI Governance Architecture

• Define identity architecture for AI workloads - Managed Identity strategy for model access and RBAC for AI resource governance.
• Design private connectivity for AI services - ensuring inference traffic does not traverse the public internet.
• Architect network segmentation between inference, training, and agent orchestration workloads.
• Design infrastructure-level Responsible AI controls and AI guardrails - content filtering integration points, prompt-injection mitigation, and data residency controls.
• Define model governance standards - model registry architecture, version control, promotion gates, and audit trails for deployed models and agents.

7. Resilience, Scalability & Cost Optimisation

• Design multi-region resilience patterns for AI workloads - endpoint failover, model endpoint high availability, and project replication.
• Architect autoscaling for inference workloads, accounting for GPU cost sensitivity and cold-start latency.
• Define capacity and quota management across subscriptions for shared AI services.
• Apply Well-Architected cost optimisation principles to AI infrastructure - rightsizing compute, reserved capacity planning, GPU utilisation efficiency, and token-level consumption awareness.

8. AI Observability, Governance & MLOps Architecture

• Define the technical governance framework - model registry architecture, endpoint lifecycle management, and model governance standards for deployed models and agents.
• Architect the MLOps/LLMOps foundation - CI/CD for model and agent deployment using Terraform and native hyperscaler tooling, prompt versioning, and evaluation pipelines.
• Design AI observability architecture - token usage, per-inference latency, model drift signals, GPU utilisation monitoring, and hallucination-tracking hooks.
• Define enterprise standards for how agents are hosted, secured, and connected to systems.
• Provide architecture design review for AI use cases proposed by business units, ensuring infrastructure is scalable, reusable, and aligned to enterprise standards.
• Track hyperscaler AI roadmaps (AWS, Azure, GCP) and proactively incorporate emerging capabilities into the reference architecture.

Practice Building

• Codify methods and frameworks into reusable AI infrastructure assets for use across engagements.
• Create differentiated infrastructure offerings, accelerators, and reference implementations.
• Mentor team members in AI infrastructure architecture.
• Support engagement budgets, forecasting, and financial proposals.

Qualifications

Qualifications & Certifications

Educational Background

Bachelor’s degree in Computer Science, Information Technology, Engineering, or a closely related technical discipline. A Master’s degree is preferred.

Mandatory

• AWS Certified Solutions Architect – Professional, OR
• Microsoft Certified: Azure Solutions Architect Expert (AZ-305), OR
• Google Cloud Professional Cloud Architect

Preferred

• AWS Machine Learning Specialty, OR Azure AI Engineer Associate, OR Google Professional Machine Learning Engineer
• Azure OpenAI Service / AWS Bedrock / GCP Vertex AI specialisation

Key Competencies & Skills

Competencies

FUNCTIONAL COMPETENCIES

• Technical stakeholder advisory - advise client architects and technology leads on AI infrastructure strategy and architectural trade-offs.
• Architecture strategy and roadmap development - assess current-state environments, identify gaps, define target-state designs, and build pragmatic modernisation roadmaps.
• Structured problem-solving under ambiguity - frame AI infrastructure challenges, evaluate trade-offs, and produce actionable recommendations.
• Practice building - codify expertise into reusable frameworks, accelerators, and playbooks.

TECHNICAL COMPETENCIES

• Hyperscaler AI infrastructure - deep expertise in at least one of: AWS (Bedrock, SageMaker, EKS, EC2 GPU), Azure (AI Foundry, Azure OpenAI, Azure ML, AKS), or GCP (Vertex AI, Gemini, GKE, Cloud Run). Working knowledge of a second hyperscaler is a strong advantage.
• AI platform architecture - Azure AI Foundry, AWS Bedrock, GCP Vertex AI; model catalog governance, lifecycle management, and self-service enablement patterns.
• Container orchestration - Kubernetes (AKS, EKS, GKE), Docker, and cloud-native infrastructure design for AI workloads.
• Infrastructure as code - Terraform, Ansible, and native hyperscaler IaC tooling; able to set IaC governance standards.
• Inference layer architecture - model serving, routing, caching, latency budgets, and cost optimisation for real-time and batch inference.
• Retrieval and RAG architecture - vector databases, embedding infrastructure, hybrid search, GraphRAG, and multi-tenant isolation at enterprise scale.
• Agentic architecture - tool calling infrastructure, MCP server hosting, agent orchestration frameworks, multi-agent systems, and agent memory patterns.
• MLOps/LLMOps architecture - CI/CD for models, pipeline automation, model versioning, model governance, and evaluation pipelines.
• AI governance and Responsible AI - model governance frameworks, AI guardrails, Managed Identity, RBAC, private connectivity, and Responsible AI controls as applied to AI workloads.
• AI observability - token usage monitoring, per-inference latency, model drift signals, GPU utilisation tracking, and cost attribution.
• Cloud cost optimisation - Well-Architected cost principles applied to AI workloads; compute rightsizing, reserved capacity, GPU utilisation efficiency, and consumption governance.
• AI Compute Infrastructure - familiarity with NVIDIA GPU ecosystem, NVIDIA NIM, DGX platforms, GPU sizing strategies, AI accelerator technologies, and enterprise AI deployment patterns.

Additional Information

Equal Opportunities

Accenture is an equal opportunities employer and welcomes applications from all sections of society and does not discriminate on grounds of race, religion or belief, ethnic or national origin, disability, age, citizenship, marital, domestic or civil partnership status, sexual orientation, gender identity, or any other basis as protected by applicable law.

Experience

10–14 years of combined experience across enterprise architecture, cloud infrastructure, and/or AI infrastructure delivery.

• 3+ years designing end-to-end AI infrastructure architectures with architecture ownership.
• Strong expertise in at least ONE hyperscaler (AWS, Azure, or GCP) is mandatory; second hyperscaler is a strong advantage.
• Hands-on design experience with Kubernetes (AKS, EKS, or GKE) and container orchestration at enterprise scale.
• Proficiency in IaC tools (Terraform, Ansible) and native hyperscaler CLIs.
• Experience designing inference serving patterns - provisioned vs pay-as-you-go, model routing, and self-hosted inference on GPU compute.
• Production-scale architectural experience with at least one vector database - sizing, indexing strategy, and multi-tenant isolation.
• Experience designing tool calling and MCP architecture for agentic systems, including security and governance of tool access.
• Experience with multi-agent orchestration frameworks (Semantic Kernel, LangGraph, CrewAI) from an infrastructure-hosting perspective.
• Understanding of MLOps/LLMOps practices - CI/CD for ML, model versioning, model governance, pipeline automation.
• Knowledge of AI observability stacks (Prometheus, Grafana, ELK, OpenTelemetry) and reliability engineering practices.
• Understanding of Well-Architected cost optimisation principles - compute rightsizing, reserved capacity, GPU utilisation efficiency, and consumption governance.
• Understanding of Responsible AI principles and AI guardrails at the infrastructure level.
• Enterprise Architecture knowledge (TOGAF or equivalent).
• Architecture design skills at L1–L3 levels; proficiency in draw.io, Visio, or Lucid Chart.

About Accenture

Accenture is a leading global professional services company that helps the world’s leading businesses, governments and other organizations build their digital core, optimize their operations, accelerate revenue growth and enhance citizen services—creating tangible value at speed and scale. We are a talent- and innovation-led company with approximately 791,000 people serving clients in more than 120 countries. Technology is at the core of change today, and we are one of the world’s leaders in helping drive that change, with strong ecosystem relationships. We combine our strength in technology and leadership in cloud, data and AI with unmatched industry experience, functional expertise and global delivery capability. Our broad range of services, solutions and assets across Strategy & Consulting, Technology, Operations, Industry X and Song, together with our culture of shared success and commitment to creating 360° value, enable us to help our clients reinvent and build trusted, lasting relationships. We measure our success by the 360° value we create for our clients, each other, our shareholders, partners and communities.

Visit us atwww.accenture.com

Equal Employment Opportunity Statement

We believe that no one should be discriminated against because of their differences. All employment decisions shall be made without regard to age, race, creed, color, religion, sex, national origin, ancestry, disability status, militaryveteran status, sexual orientation, gender identity or expression, genetic information, marital status, citizenship status or any other basis as protected by applicablelaw. Our rich diversity makes us more innovative, more competitive, and more creative, which helps us better serve our clients and our communities.