Generative AI and Human Decision-Making Behavior in Analytic-Evaluative Tasks: A Scoping Review (PRISMA-ScR)

Abstract

This project contains the pre-registered protocol and supplementary materials for a scoping review examining how generative artificial intelligence (GenAI) influences human decision-making behavior in analytic-evaluative tasks. The review follows the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) reporting guideline and the methodological frameworks of Arksey and O'Malley as refined by Levac et al. and is guided by the JBI methodology for scoping reviews. The search strategy covers three complementary sources — Scopus, Web of Science Core Collection, and Google Scholar (via Harzing's Publish or Perish) — supplemented by forward and backward citation searching and targeted journal scans; the final deduplicated corpus (n = 11,329 records) was frozen on 2026-07-29 and is documented in the corpus build log. The search strategy underwent a PRESS 2015 self-assessment and was validated against a frozen benchmark reference set prior to registration. Screening follows a human-in-the-loop (HitL) paradigm: all inclusion decisions are made by a human reviewer; AI-based tools serve exclusively as decision support. Stage 1 (title/abstract screening) was calibrated and validated using human-coded reference sets with one-sided exact Clopper–Pearson confidence bounds and predefined GO/NO-GO criteria; the GO decision was reached on 2026-08-06 and the Stage-1 prompt was version-frozen. This registration is submitted after corpus freeze and completed Stage-1 validation, and prior to the Stage-1 full screening run, Stage-2 full-text screening, and data charting. Stage-2 procedures — including the calibration threshold of at least 30 human-coded includes, the HitL review workflow, and the operationalisation of data charting depth — are pre-registered in the attached protocol and amendment packages. The registered protocol is version 1.9 (dated 2026-08-13; SHA256 hash documented in the integrity manifest), a structural revision of the supervisor-approved v1.8 affecting only data-deposit statements, with versions 1.6 to 1.8 retained as version history. File integrity of the complete registration package is verifiable via the included SHA256 manifest (MANIFEST_v1_9_EN.csv; 92 entries: 61 public package files and 31 restricted audit-trail files, the latter documented by hash only). Post-registration artefacts (Stage-2 runs, calibration reports, full-text exclusion lists, PRISMA-ScR flow numbers) will be added to this project with their own hashes and dated log entries, without altering any frozen registration file. Files containing third-party bibliographic records with abstracts (the benchmark reference set, calibration and validation coding sheets, and raw screening model outputs) are not deposited in this OSF project for copyright and database-license reasons. They are held in a locally versioned, integrity-secured repository maintained by the review author; their SHA256 hashes are publicly documented in the integrity manifest (MANIFEST_v1_9_EN.csv), so the completeness and integrity of the full registration package remain publicly verifiable. Access can be granted to editors, reviewers, or researchers upon reasonable request. Constructs of interest include advice taking (Weight of Advice, Judge–Advisor System), trust and reliance behavior (trust, reliance, overreliance), algorithm aversion and appreciation, cognitive biases (anchoring, automation bias), delegation, and human–AI decision performance, alongside cognitive load in human–GenAI interaction. This scoping review constitutes the first study of a cumulative doctoral dissertation supervised by Prof. Dr. Frank Teuteberg (Universität Osnabrück).