Artificial Intelligence Based Framework for Student Engagement Assessment in Classroom Environments

Abstract

Student distraction remains a critical barrier to effective learning in classroom environments, yet existing detection methods relying on manual observation or post-session feedback are inherently subjective, time-intensive, and ill-suited for real-time adaptive teaching. This study presents the Artificially Intelligent Technology for Student Engagement Assessment in Classroom Environments (AI-TEACH), a novel co-teacher paradigm that automatically identifies and quantifies student distraction through multimodal fusion of asynchronized audio and video streams captured via classroom surveillance cameras. AI-TEACH integrates YOLO-NAS and ByteTrack for real-time student detection and tracking, MediaPipe for behavioral cue extraction, and Silero-VAD with Wav2Vec2-SVM for audio-based emotional distraction classification; a BiLSTM network fuses these multimodal features to generate per-incident severity scores, and a session-wide engagement report accessible through an interactive instructor dashboard. The framework was validated on a curated dataset of classroom recordings and further evaluated through a controlled experiment involving 200 students divided into experimental and control groups, achieving 90% accuracy and 92% recall in distraction detection. Compared with existing unimodal and non-real-time approaches, AI-TEACH offers a more accurate, scalable, and immediately actionable solution for objective student engagement monitoring in modern classroom practice, with direct implications for data-informed pedagogy and inclusive education.