The paper develops the Fake Friend Dilemma as a framework for understanding conflicts between users and the institutions shaping conversational AI. It identifies two central conditions: users placing relational trust in a system, and the system being influenced by institutional interests that do not match the user’s interests. That trust may encourage disclosure, reliance on advice and receptivity to recommendations.

The authors examine how the dilemma could arise in product sales, propaganda and biased information, surveillance, and behavioral nudging. They also discuss possible responses, including clearer explanations, reminders about data collection and ownership conflicts, and technical safeguards intended to align systems more closely with users.