How Good Is Your Forecast? The Importance of User‐Centred, Decision‐Relevant Evaluation of Forecasts and Climate Services

Abstract

ABSTRACT Forecasts and climate services, spanning weather and climate timescales, play a vital role in supporting short‐ and long‐term decision‐making across a wide range of climate‐sensitive sectors. Understanding their skill is crucial to ensuring that the information provided is relevant, trustworthy and usable in real‐world contexts. Traditional evaluation and verification methods typically focus on statistical accuracy, which remains essential for model development and improvement. However, these metrics alone may not capture the aspects of forecast performance that matter most to users and often fail to reflect how well forecasts could support practical decision‐making. This paper highlights the importance of user‐driven, decision‐relevant evaluation: an approach that centres on the needs, experiences and decision‐making contexts of forecast and climate service users. We present a flexible framework for user‐driven forecast evaluation. Drawing on case studies from diverse regions and hazard types—including cold waves in Lesotho, precipitation variability in Spain and drought in Lesotho, the Netherlands, Italy and Georgia—we demonstrate how user engagement can tailor evaluation to decision‐making contexts and local knowledge. The framework combines event‐based contingency table approaches, using user‐defined thresholds that are linked to the decision they make, to derive transparent metrics such as hits, misses and false alarms, alongside scores such as probability of detection, false alarm ratio and threat score. Traditional verification metrics (such as mean absolute error or the continuous ranked probability score) are applied as complementary diagnostics, while co‐evaluation of forecast performance for historical events is used to further support dialogue, accessibility and trust. The case studies show the value of combining long‐term performance metrics with context‐rich narratives and emphasise the importance of data visualisation and communication in shaping forecast usability. We conclude with recommendations for integrating user‐centred evaluation into operational workflows to ensure climate services are both scientifically robust and decision‐relevant.