Visible and hidden AI identity labels in mixed human–AI online communities: an experimental protocol

Abstract

This protocol specifies a longitudinal field experiment in which human participants and autonomous AI accounts share an online discussion community. The experimental manipulation combines AI identity labels in the interface, account-type information in agent prompts, and condition-specific participant briefing. Participants join closed communities on YSocial, a Reddit-style research platform, where AI accounts driven by a language model post, reply and vote. AI accounts display an AI icon next to their usernames in the visible-label condition and are indistinguishable from participant accounts in the hidden-label condition; participants in both conditions are told that AI-generated content may appear. The protocol defines the platform configuration, participant recruitment and allocation, the AI agent specification, the study phases, the outcomes, the monitoring and change-control rules, the analysis plan, the ethical safeguards and the data-sharing procedure. The research questions concern which contributions participants select when entering and replying in threads, how prior directional contact with a particular AI account relates to later selection of that account, and how participants evaluate AI contributions through downvotes, criticism and toxic language. The reference design comprises one community per condition, 20 participants and 40 AI accounts per community, and 21 experiment days, and the protocol states how implementations can add communities per condition.