AI Governance & Evaluation Analyst

WHY UT SOUTHWESTERN?
With over 75 years of excellence in Dallas-Fort Worth, Texas, UT Southwestern is committed to excellence, innovation, teamwork, and compassion. As a world-renowned medical and research center, we strive to provide the best possible care, resources, and benefits for our valued employees. Ranked as the No. 1 hospital in Dallas-Fort Worth according to U.S. News & World Report, we invest in you with opportunities for career growth and development to align with your future goals. Our highly competitive benefits package offers healthcare, PTO and paid holidays, on-site childcare, wage, merit increases and so much more. We invite you to be a part of the UT Southwestern team where you’ll discover a culture of teamwork, professionalism, and a rewarding career!

JOB SUMMARY
The responsible Artificial Intelligence (AI) Governance & Evaluation Analyst supports UT Southwestern's AI governance program by managing the review, evaluation, and monitoring of AI use cases across healthcare, research, education, and administrative functions. The role works closely with project teams, subject matter experts, governance committees, and institutional partners to ensure AI systems are reviewed, documented, monitored, and deployed responsibly.

BENEFITS
UT Southwestern is proud to offer a competitive and comprehensive benefits package to eligible employees. Our benefits are designed to support your overall wellbeing, and include\:

• PPO medical plan, available day one at no cost for full-time employee-only coverage
• 100% coverage for preventive healthcare-no copay
• Paid Time Off, available day one
• Retirement Programs through the Teacher Retirement System of Texas (TRS)
• Paid Parental Leave Benefit
• Wellness programs
• Tuition Reimbursement
• Public Service Loan Forgiveness (PSLF) Qualified Employer
• Learn more about these and other UTSW employee benefits!

EXPERIENCE AND EDUCATION

Required

• Education

Master's Degree in Public Health, Biomedical Informatics, Data Science, Computer Science, Engineering, Research Administration, or a related field.

• Experience

5 years of professional experience in AI/ML, healthcare operations, clinical informatics, research administration, data analytics, technology governance, compliance, quality, program management, or a related area.
Experience coordinating complex projects, reviews, or governance processes involving multiple stakeholders.

Preferred

• Education

PhD or other advanced degree in a related field.

• Experience

Experience in healthcare, academic medicine, research, life sciences, or other regulated environments.
Experience with AI governance, AI evaluation, validation, assurance, vendor review, digital health, or lifecycle monitoring.
Familiarity with responsible AI frameworks and standards such as NIST AI RMF and ISO 42001 preferred.

JOB DUTIES

• AI Governance & Case Management
  • Manage intake, triage, tracking, and documentation of AI use cases.
  • Guide teams through governance requirements and review processes.
  • Coordinate reviews, approvals, monitoring activities, and reassessments.
  • Maintain accurate, complete, and audit-ready records.
  • Escalate high-risk or complex issues when appropriate.
• AI Evaluation & Assurance
  • Coordinate AI evaluations using approved standards and methodologies.
  • Collect and organize evidence related to performance, safety, fairness, privacy, security, and reliability.
  • Support validation, testing, monitoring, and assurance activities.
  • Document findings, mitigation plans, and monitoring requirements.
  • Track post-deployment performance and report significant issues.
• Governance Committee Support
  • Prepare agendas, case summaries, and review materials.
  • Coordinate meetings, reviewers, and subject matter experts.
  • Document decisions, action items, and follow-up activities.
  • Communicate outcomes and track completion of required actions.
• Portfolio & Vendor Oversight
  • Maintain records of AI systems, vendors, approvals, risk classifications, and monitoring requirements.
  • Coordinate AI-related vendor due diligence with relevant departments.
  • Monitor changes to AI systems and ensure reassessment when necessary.
  • Maintain templates, evidence repositories, and governance documentation.
• Generative & Agentic AI Governance
  • Support governance activities for large language models, foundation models, multimodal AI, and agentic AI systems.
  • Review information related to model performance, oversight, access controls, guardrails, and testing.
  • Track model changes and document limitations and monitoring requirements.
• Partnership & Stakeholder Support
  • Serve as a primary point of contact for assigned AI initiatives.
  • Work collaboratively with clinical, research, technical, compliance, legal, privacy, and operational teams.
  • Provide guidance, training, consultations, and office-hour support.
  • Promote consistent governance practices across the organization.
• Reporting & Continuous Improvement
  • Monitor governance metrics, workload, review timelines, and stakeholder experience.
  • Prepare dashboards, reports, and presentations for leadership.
  • Identify process improvements and support development of procedures, templates, and training materials.
  • Stay informed on emerging AI technologies, regulations, and industry standards.

Knowledge, Skills and Abilities

• Strong organizational, project coordination, and documentation skills.
• Ability to communicate complex technical and regulatory information clearly.
• Strong relationship-building, facilitation, and customer service skills.
• Ability to manage multiple priorities and deadlines.
• Working knowledge of AI technologies, including machine learning and generative AI.
• Ability to handle confidential information with discretion.

Salary Information
Salary is commensurate with education, skills, and relevant experience.

SECURITY AND EEO STATEMENT

Security
This position is security-sensitive and subject to Texas Education Code 51.215, which authorizes UT Southwestern to obtain criminal history record information.

EEO
UT Southwestern Medical Center is committed to an educational and working environment that provides equal opportunity to all members of the University community. As an equal opportunity employer, UT Southwestern prohibits unlawful discrimination, including discrimination on the basis of race, color, religion, national origin, sex, sexual orientation, gender identity, gender expression, age, disability, genetic information, citizenship status, or veteran status.WHY UT SOUTHWESTERN?
With over 75 years of excellence in Dallas-Fort Worth, Texas, UT Southwestern is committed to excellence, innovation, teamwork, and compassion. As a world-renowned medical and research center, we strive to provide the best possible care, resources, and benefits for our valued employees. Ranked as the No. 1 hospital in Dallas-Fort Worth according to U.S. News & World Report, we invest in you with opportunities for career growth and development to align with your future goals. Our highly competitive benefits package offers healthcare, PTO and paid holidays, on-site childcare, wage, merit increases and so much more. We invite you to be a part of the UT Southwestern team where you’ll discover a culture of teamwork, professionalism, and a rewarding career!

JOB SUMMARY
The responsible Artificial Intelligence (AI) Governance & Evaluation Analyst supports UT Southwestern's AI governance program by managing the review, evaluation, and monitoring of AI use cases across healthcare, research, education, and administrative functions. The role works closely with project teams, subject matter experts, governance committees, and institutional partners to ensure AI systems are reviewed, documented, monitored, and deployed responsibly.

BENEFITS
UT Southwestern is proud to offer a competitive and comprehensive benefits package to eligible employees. Our benefits are designed to support your overall wellbeing, and include\:

• PPO medical plan, available day one at no cost for full-time employee-only coverage
• 100% coverage for preventive healthcare-no copay
• Paid Time Off, available day one
• Retirement Programs through the Teacher Retirement System of Texas (TRS)
• Paid Parental Leave Benefit
• Wellness programs
• Tuition Reimbursement
• Public Service Loan Forgiveness (PSLF) Qualified Employer
• Learn more about these and other UTSW employee benefits!

EXPERIENCE AND EDUCATION

Required

• Education

Master's Degree in Public Health, Biomedical Informatics, Data Science, Computer Science, Engineering, Research Administration, or a related field.

• Experience

5 years of professional experience in AI/ML, healthcare operations, clinical informatics, research administration, data analytics, technology governance, compliance, quality, program management, or a related area.
Experience coordinating complex projects, reviews, or governance processes involving multiple stakeholders.

Preferred

• Education

PhD or other advanced degree in a related field.

• Experience

Experience in healthcare, academic medicine, research, life sciences, or other regulated environments.
Experience with AI governance, AI evaluation, validation, assurance, vendor review, digital health, or lifecycle monitoring.
Familiarity with responsible AI frameworks and standards such as NIST AI RMF and ISO 42001 preferred.

JOB DUTIES

• AI Governance & Case Management
  • Manage intake, triage, tracking, and documentation of AI use cases.
  • Guide teams through governance requirements and review processes.
  • Coordinate reviews, approvals, monitoring activities, and reassessments.
  • Maintain accurate, complete, and audit-ready records.
  • Escalate high-risk or complex issues when appropriate.
• AI Evaluation & Assurance
  • Coordinate AI evaluations using approved standards and methodologies.
  • Collect and organize evidence related to performance, safety, fairness, privacy, security, and reliability.
  • Support validation, testing, monitoring, and assurance activities.
  • Document findings, mitigation plans, and monitoring requirements.
  • Track post-deployment performance and report significant issues.
• Governance Committee Support
  • Prepare agendas, case summaries, and review materials.
  • Coordinate meetings, reviewers, and subject matter experts.
  • Document decisions, action items, and follow-up activities.
  • Communicate outcomes and track completion of required actions.
• Portfolio & Vendor Oversight
  • Maintain records of AI systems, vendors, approvals, risk classifications, and monitoring requirements.
  • Coordinate AI-related vendor due diligence with relevant departments.
  • Monitor changes to AI systems and ensure reassessment when necessary.
  • Maintain templates, evidence repositories, and governance documentation.
• Generative & Agentic AI Governance
  • Support governance activities for large language models, foundation models, multimodal AI, and agentic AI systems.
  • Review information related to model performance, oversight, access controls, guardrails, and testing.
  • Track model changes and document limitations and monitoring requirements.
• Partnership & Stakeholder Support
  • Serve as a primary point of contact for assigned AI initiatives.
  • Work collaboratively with clinical, research, technical, compliance, legal, privacy, and operational teams.
  • Provide guidance, training, consultations, and office-hour support.
  • Promote consistent governance practices across the organization.
• Reporting & Continuous Improvement
  • Monitor governance metrics, workload, review timelines, and stakeholder experience.
  • Prepare dashboards, reports, and presentations for leadership.
  • Identify process improvements and support development of procedures, templates, and training materials.
  • Stay informed on emerging AI technologies, regulations, and industry standards.

Knowledge, Skills and Abilities

• Strong organizational, project coordination, and documentation skills.
• Ability to communicate complex technical and regulatory information clearly.
• Strong relationship-building, facilitation, and customer service skills.
• Ability to manage multiple priorities and deadlines.
• Working knowledge of AI technologies, including machine learning and generative AI.
• Ability to handle confidential information with discretion.

Salary Information
Salary is commensurate with education, skills, and relevant experience.

SECURITY AND EEO STATEMENT

Security
This position is security-sensitive and subject to Texas Education Code 51.215, which authorizes UT Southwestern to obtain criminal history record information.

EEO
UT Southwestern Medical Center is committed to an educational and working environment that provides equal opportunity to all members of the University community. As an equal opportunity employer, UT Southwestern prohibits unlawful discrimination, including discrimination on the basis of race, color, religion, national origin, sex, sexual orientation, gender identity, gender expression, age, disability, genetic information, citizenship status, or veteran status.