Director AI Evaluation

Location

Work from Home Job Category Business Strategy and Innovations Schedule Days Work Type Full time Department Artificial Intelligence Division Date posted 07/16/2026 Job ID R-99357 ### Job Summary

The Director of AI Evaluation owns how Geisinger defines, proves, and sustains quality across its entire AI portfolio — internally built models and vendor-provided systems alike. This is a hands-on technical leader who also manages the people who do the building and the proving: the data scientists who develop production machine learning and fine-tuned AI systems, and the senior analysts who evaluate them. Every high-value AI initiative — bought or built — needs a single, credible standard for what constitutes quality, who validates it, and how it stays good in production. The Director sets that standard, leads the team that enforces it, and reports findings to the VP of AI, executive leaders, and the board. This role is a manager who develops a multidisciplinary team, a technical authority who defines evaluation method across the enterprise, and a quality owner who guides every major AI program toward evidence that withstands scrutiny.

### Job Duties

• Reports to the VP of AI; directly career-manages the data science line and matrix-manages the Senior Analysts, AI Evaluation.
• Determines the quality standard for any high-value AI initiative at Geisinger — internally built or vendor-provided — from design through production.
• Holds bought systems to the same standard as built ones, generating local evidence on whether a tool works here for Geisinger's clinicians and patients rather than accepting vendor aggregate or cherry-picked results.
• Owns the methodology that holds initiatives to that standard: pre-production validation and live production monitoring.
• Owns the health of the data science team — attracting and retaining strong technical talent, developing careers, and keeping the bench deep, eng