Don’t Blame the Code! AI Paternalism Reconsidered

Abstract

Abstract AI-driven diagnostic and therapeutic systems are being rapidly integrated into clinical practice. A growing literature contends that this integration threatens patient autonomy by exposing patients to a novel form of AI paternalism. I argue that this framing is misguided on both theoretical and practical grounds. Theoretically, AI systems cannot be paternalists: their outputs lack the normative and agential standing that paternalism requires – a limitation I articulate as the is–ought gap for AI-generated output . Any paternalistic force in AI-integrated care must therefore reside in human agents, not in the systems themselves. Hence, we should not blame the code for any paternalism that patients may be subjected to in AI-integrated healthcare, but rather the human agents who purpose and develop these systems. In addition to these theoretical and conceptual concerns, there are practical worries. Casting AI integration as a case of paternalism risks framing the use of AI as prima facie objectionable and channels the debate toward a narrow, binary remedy that leans heavily on algorithmic transparency as a prerequisite for informed consent and shared decision-making. These discussions, I argue, are better served by revising our underlying conception of autonomy itself. I therefore conclude this article by arguing that replacing traditional, individualist conceptions of autonomy with a relational account better captures the distributed, socially embedded character of decision-making in AI-integrated care. This also redirects normative attention from the prevention of paternalism toward the conditions under which patient autonomy can be actively enabled and promoted.