AI-in-the-human-loop: an operational model for mature human-AI collaboration

Abstract

Abstract Artificial intelligence (AI) and large language models are powerful tools but come with risks and caveats for proper usage. In this work, we are concerned with effective human-AI collaboration. As in any team, it is key that all team members properly understand the problem at hand, the environment, and each other. In order to scaffold a mature human-AI collaboration, we propose a process framework based on continual, proactive, bidirectional assessment, fostering co-evolution of the human-AI team. While most research has focused on assessing the strengths and limitations of AI agents, we especially focus on the risks associated with a lack of maturity on the human side. The core idea is to have AI agents help humans assess themselves and to anticipate interaction issues. Furthermore, to place the proposal within a broader vision, we advance two novel concepts: (i) “AI-in-the-human-loop”, the idea of AI agents observing and acting within human reasoning and activity, and (ii) “human-under-test”, a software testing analogy that suggests techniques for assessing human maturity by AI agents. A case study of a human-AI team involved in a statistical analysis project is considered, and analysed through an LLM-as-a-judge setup. Preliminary results highlight collaboration maturity issues and the possibility of proactive corrections, showcasing limitations in current collaborative workflows and opportunities for further research.