Hype vs. reality: the true impact of traditional and generative AI on users’ intention to use soft skills coaching chatbots in higher education

Abstract

This study compares three soft-skills coaching chatbot configurations in higher education: traditional AI, hybrid AI, and generative AI-only systems. In collaboration with Rocky.ai, a prominent European AI coaching chatbot company, we investigate a sample of 120 higher education business students and their acceptance and behavioural intention to use these systems for the development of soft skills. The study builds on the Unified Theory of Acceptance and Use of Technology (UTAUT) by incorporating Perceived Risk (PR) and Perceived Trust (PT) and by examining additional mediational relationships among core constructs. The findings indicate that PT and Performance Expectancy (PE) are the most consequential explanatory mechanisms of behavioural intention across the compared models, whereas PR plays a more limited role under the present design and measurement approach. The comparison across chatbot configurations further suggests that users do not evaluate traditional, hybrid, and generative AI-only systems through identical acceptance dynamics. Rather than supporting the mainstream assumption that newer generative systems are necessarily better received, the results indicate that, in this educational coaching context, behavioural intention depends strongly on how users assess expected usefulness and trustworthiness. This study contributes to the chatbot literature, and particularly to the nascent coaching chatbot literature, by clarifying how trust, PE, and system configuration interact in shaping users’ intention to use AI-based coaching systems in higher education.