Expert evaluation of AI-assisted scenario design for interactive problem based learning environments in medical education

Abstract

Problem-based learning (PBL) is an interactive learning approach in which students develop understanding through collaborative inquiry, hypothesis generation, and interpretation of clinical information. Because scenario quality shapes reasoning and self-directed learning, this study examined whether AI-assisted scenario generation could support the development of high-quality PBL materials in medical education. Using a paired comparative quantitative design, human-written and ChatGPT-generated scenarios addressing identical learning objectives were compared. Twenty scenarios, organized as 10 matched pairs, were evaluated by independent academic experts using a 24-item checklist refined through pilot expert review. Across valid ratings, AI-generated scenarios received a higher proportion of positive evaluations than human-written scenarios (90.7% vs 80.3%), particularly for clarity, consistency, cognitive load balance, scientific accuracy, and support for differential diagnosis. Significant differences were observed in selected matched pairs after adjustment for multiple comparisons. Generative AI, when guided by explicit pedagogical principles and followed by expert review, may therefore serve as a useful co-design tool for PBL scenario development. Human academic judgment remains essential for psychosocial nuance, cultural sensitivity, and contextual depth.