Advancing evaluation methodologies for CBRN simulation exercises: integrating social psychology with program evaluation

Abstract

Simulation exercises are widely used to rehearse plans, test capabilities, clarify interdependencies, and identify preparedness gaps for chemical, biological, radiological, and nuclear (CBRN) hazards. Existing procedural evaluation guidance is useful, but procedural indicators alone may not show whether participants develop shared situation awareness, understand authorities, responsibilities, and role interdependencies, coordinate across agencies, adapt as scenarios evolve, or trust partner organizations enough to act on their information. This conceptual synthesis integrates program evaluation theory, social psychological theory, authoritative exercise guidance, and peer-reviewed exercise and preparedness studies to advance a modular framework for CBRN simulation-exercise evaluation. The framework organizes evaluation around five non-exhaustive domains: team cognition and shared situation awareness; role clarity and confidence in role performance; communication and coordination dynamics; behavioral and cognitive adaptation under uncertainty; and inter-organizational trust and collaborative orientation. For each domain, the article links theoretical constructs to observable indicators and feasible mixed-method data sources, including structured behavioral observation, brief participant surveys, cognitive walkthrough interviews, and structured debriefs. The proposed evaluation architecture complements rather than replaces procedural evaluation protocols by helping evaluators explain why critical tasks were or were not performed and what findings imply for planning, training, governance, and organizational design. Used selectively according to exercise purpose, tested capabilities, scope, exercise type, available resources, and primary intended users, the framework is intended to support more diagnostic and actionable CBRN exercise evaluation.