AI Prompt Engineer 2

About the Role

We are seeking an AI Prompt Engineer II who combines strategic thinking, scientific expertise, and hands-on building skills. You will join a dynamic, agile team dedicated to designing, testing, and scaling large language model (LLM) prompts that drive impactful user experiences. Leveraging your deep knowledge of Generative AI, you will enhance our AI-powered expert platform, unlocking meaningful business outcomes and shaping how AI understands customers and supports experts.

What You’ll Do

• Design, build, and develop AI agents
• Create complex multi-part prompting systems with parameter optimization and implement advanced prompting techniques
• Develop comprehensive evaluation frameworks to measure performance across multiple dimensions, comparing approaches such as retrieval-augmented generation (RAG), fine-tuning, and advanced prompting methodologies
• Analyze and optimize LLM performance using A/B testing, usage metrics, and strategic feedback loops
• Develop AI-driven workflows that improve user interactions and Customer Care support
• Design and refine expert playbooks to enhance knowledge sharing and operational efficiency
• Run experiments to validate ideas rapidly and iterate based on performance data and user insights
• Collaborate cross-functionally with product managers, engineers, and stakeholders to identify LLM opportunities and translate technical capabilities into business value
• Manage stakeholder expectations and communicate complex technical concepts effectively to both technical and non-technical audiences
• Implement LLMOps best practices including model governance, lifecycle management, and performance monitoring to ensure production-ready AI systems
• Educate and empower internal teams to build better prompts and leverage AI tools effectively

What We’re Looking For

• Master’s or PhD in Computer Science, Computational Linguistics, Machine Learning, or a related field preferred
• Strong proficiency in Python with hands-on experience building and testing complex prompting systems
• Deep understanding of LLMs and advanced frameworks, with expertise across multiple LLM vendors and model versions
• Proven experience developing and applying evaluation frameworks to test model outputs
• Production experience with RAG and fine-tuning models
• Demonstrated ability to automate large-scale API interactions and scripting for prompt testing, minimizing manual processes
• Experience with LLMOps practices, including model governance, version control, performance monitoring, and lifecycle management
• Strong stakeholder management skills and ability to collaborate effectively across technical and business teams
• Proven track record of translating complex technical concepts into actionable business insights for diverse audiences
• Analytical mindset with experience using Excel, Power BI, or Google Analytics
• Solid understanding of NLP techniques
• Excellent judgment with the ability to prioritize and independently drive projects forward while managing multiple stakeholder relationships
• Experience working in agile, sprint-based development environments; familiarity with tools such as Azure Boards or Asana