Researcher-Driven Insights for Human-Agent Teaming Testbeds

Abstract

Human-Agent Teaming (HAT) is becoming increasingly important as AI systems take on collaborative roles. However, HAT research depends on experimental testbeds to study human-agent teams. This research builds on prior work that cataloged existing HAT testbeds by identifying and characterizing user needs for future HAT testbeds. In this study, nine HAT researchers were interviewed about their experimental design practices and testbed usage. Affinity diagramming was used to identify themes about pain points with existing testbeds and user needs for future testbeds. Participants indicated growing interest in using AI technologies in testbeds, to design experiments or serve as realistic teammates. Participants emphasized the importance of agent-agnostic testbeds that enable integration with external agents. These findings inform requirements for more flexible HAT testbeds that address current limitations and accommodate future research directions.