The Modality Principle in Immersive and Non‐Immersive Virtual Reality: An Eye Tracking Study

Abstract

ABSTRACT Background According to the modality principle of the Cognitive Theory of Multimedia Learning, learning is better facilitated when visual content is accompanied by a vocal narration compared to text explanation. However, the application of this principle in immersive virtual reality (IVR) remains unclear. Objectives The present experiment aimed to evaluate the effect of immersion on the application of the modality principle by comparing desktop virtual reality with immersive virtual reality. Methods A total of 105 seventh‐grade pupils, age from 12 to 13 years old, followed a 10‐min system‐paced lesson on forest development. The lesson was presented either in immersive virtual reality or in desktop VR, with limited and matched interactivity across media, and with verbal information delivered either as audio narration or on‐screen text. Learning outcomes were assessed across factual knowledge, conceptual knowledge, comprehension, and knowledge transfer, along with eye movements, presence, and cognitive load. Results and Conclusions Results indicate modulated modality effect on factual knowledge and improved conceptual learning on the desktop VR compared to IVR, which can be explained by fewer visual fixations on relevant areas. Students reported improved germane cognitive load in IVR when texts accompanied the illustrations. The modality principle does not seem to apply in the same way depending on the degree of immersion. The content of a virtual reality lesson should be adapted according to the type of knowledge being sought.