B

Blue Cross Blue Shield of Massachusetts

AI Solution Architect

AI Application EngineeringUpdated on Nov 15
Full-timeBoston, MA, USAIn-person

About the Job

Requirements

  • A bachelor's degree in Computer Science, Data Science, Engineering, or equivalent practical experience.
  • 8–10+ years in solution architecture, software architecture, or a senior technical role specifically in cloud architecture (GCP, AWS, or Azure).
  • 3+ years of hands-on experience designing and deploying production Generative AI systems, including Retrieval-Augmented Generation (RAG), vector databases, and semantic search.
  • Proven experience designing multi-agent systems and orchestration workflows, such as Agent Development Kit (ADK), LangGraph, CrewAI, or AutoGen.
  • Strong large language model fundamentals, including prompt engineering, embeddings, vector search, fine-tuning, and distillation.
  • Deep understanding of machine learning foundations, natural language processing, model evaluation, fine-tuning, and large language model safety and guardrails.
  • Hands-on experience designing MLOps and LLMOps pipelines, including continuous integration/deployment, data versioning, model monitoring, and governance in cloud environments.
  • Expertise in cloud-native solution design ensuring enterprise-grade scalability, performance, data privacy, and security compliance.
  • Proven track record leading architecture design reviews, establishing reference architectures, and defining best practices.
  • Ability to bridge technical and business domains, deliver executive-level technical presentations, and articulate business value to senior stakeholders.
  • Demonstrated experience providing technical mentorship to engineering teams and cross-functional technical leadership.
  • Professional Cloud Architect or AI/ML Specialty certification, such as Google Cloud Professional Cloud Architect, Machine Learning Engineer, AWS Certified Solutions Architect – Professional, or equivalent.

Responsibilities

  • Design agentic workflows on Vertex AI Agent Builder, defining multi-agent orchestration patterns, tool-use strategies, function calling, and grounding configurations using Google Search, Vertex AI Search, and custom data stores.
  • Produce solution design documents, architecture decision records, and integration patterns connecting AI agents with APIs, databases, and enterprise platforms.
  • Evaluate foundation models, including the Gemini family and open models via Model Garden, for cost, latency, accuracy, and context-window fit; design prompt strategies including chain-of-thought, few-shot, and system instructions.
  • Design retrieval-augmented generation flows using Vertex AI Search, Vector Search, and custom data stores to ground agent responses in enterprise knowledge.
  • Establish input/output validation, content filtering, hallucination mitigation, and human-in-the-loop checkpoints to support responsible and reliable agent behavior.
  • Build tracing, logging, and monitoring into agent architectures to track reasoning chains, tool-call performance, latency, cost, and quality metrics.
  • Execute proof-of-concepts to validate architectural decisions, de-risk new patterns, and demonstrate feasibility before full builds.
  • Provide architectural guidance, conduct design reviews, create reusable templates and reference architectures, and help developers ramp up on the agent platform.
  • Work with data engineering, platform engineering, Network Security, and business stakeholders to align AI solutions with enterprise standards, security requirements, and business goals.
  • Track GCP platform updates, new agent capabilities, and emerging agentic AI patterns, and share relevant insights with the team.
  • Share knowledge and showcase the team's work through internal and external technical talks.

Desired Qualifications

  • Knowledge of the healthcare payer domain.
  • Experience with multi-modal agents, including vision, audio, and document processing.
  • Experience building evaluation frameworks for agent quality, including benchmarks and LLM-as-judge patterns.

About the Company

Company size
1,001-5,000
Company stage
Not listed
Total funding
$199.6M
Headquarters
Boston, MA
Founded
1937

Recently Posted Jobs

Explore more opportunities at Blue Cross Blue Shield of Massachusetts

See more jobs