DE&a – AI/ML - Multiple Teams

Key Responsibilities

• Design and develop enterprise RAG applications using LLMs, embeddings, vector databases, and hybrid search.
• Build end-to-end document ingestion and knowledge ingestion pipelines for structured and unstructured data.
• Implement document parsing, chunking, metadata enrichment, embeddings, indexing, and retrieval strategies.
• Design and optimize semantic, vector, keyword, and hybrid search solutions.
• Develop RAG workflows incorporating query understanding, query rewriting, retrieval, reranking, context generation, and response generation.
• Work with LLMs such as Azure OpenAI, Anthropic Claude, or equivalent models.
• Develop agentic AI solutions using tools, function calling, MCP, and multi-agent/single-agent architectures where appropriate.
• Implement RAG evaluation and observability including retrieval quality, answer relevance, groundedness, hallucination detection, latency, and token/cost monitoring.
• Optimize RAG applications for accuracy, latency, scalability, and cost.
• Integrate RAG applications with enterprise systems, APIs, databases, repositories, and knowledge sources.
• Develop secure APIs and backend services for AI applications.
• Collaborate with architects, developers, business analysts, and domain experts to translate business requirements into AI solutions.
• Establish best practices around prompt engineering, context management, guardrails, security, and responsible AI.
• Troubleshoot production issues and continuously improve the AI application based on user feedback and evaluation metrics.

Required Technical Skills

Generative AI / LLM

• Strong understanding of LLMs and Generative AI
• Prompt engineering and structured prompting
• LLM inference and model selection
• Function calling / tool calling
• Context-window management
• Understanding of hallucination and grounding challenges

RAG

• Strong hands-on experience building RAG applications
• Document ingestion and preprocessing
• Chunking strategies
• Metadata design and filtering
• Embedding generation
• Vector search
• Hybrid search
• Reranking
• Query expansion / rewriting
• Retrieval optimization
• RAG evaluation

AI / Agentic Frameworks

• Experience with one or more frameworks such as:
  • LangGraph
  • Google ADK
• Experience with MCP (Model Context Protocol) is a plus.
• Understanding of agent orchestration and tool-based workflows.

Cloud & Search

• Strong experience with Microsoft Azure
• Azure OpenAI / Azure AI Foundry
• Azure AI Search or equivalent vector search platform
• Azure Blob Storage
• Azure App Service / Functions
• API Management
• Experience with AWS AI services or Amazon OpenSearch is a plus.

Programming

• Strong Python development skills
• REST API development
• Flask / FastAPI
• JSON and API integrations
• Experience with SQL and relational databases

Databases / Search

• Vector databases/search engines such as:
  • Azure AI Search
  • OpenSearch
  • PostgreSQL/pgvector
  • Pinecone
  • Elasticsearch
  • Weaviate
• Understanding of indexing and search optimization.

RAG Evaluation & Observability

Experience with AI observability and evaluation tools such as:

• Arize Phoenix
• LangSmith
• Azure AI evaluation capabilities
• RAGAS
• Custom evaluation frameworks

Knowledge of metrics such as:

• Context relevance
• Context precision/recall
• Answer relevance
• Faithfulness / groundedness
• Retrieval accuracy
• Hallucination rate
• Latency
• Token consumption
• Cost per request

Preferred / Good-to-Have Skills

• Experience with Guidewire PolicyCenter, ClaimCenter, BillingCenter, or other enterprise insurance platforms.
• Experience working with large technical documentation repositories.
• Understanding of Guidewire data models, APIs, configuration, and data dictionaries.
• Experience building AI assistants for enterprise developers.
• Experience with structured knowledge extraction from HTML, XML, JSON, PDFs, source code, database schemas, and technical documentation.
• Knowledge of enterprise security, RBAC, PII protection, and data governance.

Experience with semantic caching and 

.