Six guidelines for trustworthy, ethical and responsible automation design

Abstract

Abstract Calibrated trust ( Lee and See 2004 ) is critical for safe and seamless integration of automated systems into society. 1 Users should only rely on a system recommendation when it is actually correct and reject it when it is factually wrong. This requires an accurate trustworthiness assessment, ensuring that the user's perception of the system’s trustworthiness aligns with its actual trustworthiness, allowing users to make informed decisions about the extent to which they can rely on the system ( Schlicker et al. 2025 ). The field of pragmatics offers valuable insights into communication processes, emphasizing the context and mutual understanding. These are essential for developing accurate trustworthiness assessments, bridging the gap between user expectations and system capabilities. We propose six design guidelines to help designers foster accurate trustworthiness assessments, and thus ethical and responsible human-automation interactions. The guidelines draw on human-computer interaction, cognitive psychology, automation research, user-experience design, ethics, and pragmatics — specifically, the cultivation of common ground ( Clark 1996 ) and Gricean communication maxims ( Grice 1975 ). Pragmatic principles provide insights into these contexts, because the user’s perception of the system’s trustworthiness is shaped by environmental contexts, such as organizational culture or societal norms, and situational context, such as the specific interaction scenario ( Hoff and Bashir 2015 ). The guidelines provide actionable insights for designers to create automated systems that make relevant trustworthiness cues available, and serve as a tool for evaluating to what extent existing systems enable users to accurately assess a system’s trustworthiness.