Development and evaluation of eduvate CBT as an integrated artificial intelligence powered assessment ecosystem

Abstract

The growth of digital assessment has intensified concerns about examination integrity, grading efficiency, fairness, and responsible artificial intelligence (AI). This study evaluates Eduvate CBT, an integrated role-based assessment ecosystem combining examination administration, AI-assisted proctoring, automated objective grading, reviewable violation evidence, and learning analytics. A four-week quantitative evaluation covered 250 undergraduate examination sessions across four computing courses, with usability data from 15 instructors and 5 administrators. At the session level, proctoring achieved 94.8% accuracy (95% CI 91.3–96.9), 94.6% precision, 96.6% recall, and an F1-score of 95.6%. Automated grading agreed with instructor consensus for 742 of 750 sampled responses (98.9%; 95% CI 97.9–99.5) and processed the cohort in under 10 s, compared with an estimated 23.1 h of manual grading. The aggregate System Usability Scale score was 87.4 (SD = 5.8; 95% CI 84.7–90.1). Analytics identified six low-performing items in algorithmic complexity, demonstrating how assessment evidence can prompt instructional review. The findings support the feasibility of the integrated workflow, although generalisability is limited by the single-institution design, absence of a competing-system control, and unavailable formal inter-rater reliability estimates.