Multiuser immersive cultural experiences enabled by 5G, AI and free-viewpoint video

Abstract

Extended Reality (XR) enhances cultural experiences by immersing users in virtual environments where virtual assets and 3D reconstructions provide a stronger connection to real-world scenarios. In this work, we present a platform for collaborative immersive cultural experiences, focused on real-time interaction between participants. The platform provides three main features designed to enable meaningful communication inside of virtual scenarios: volumetric representation of a presenter captured in real time, virtual scene control through gesture recognition for the presenter, and bidirectional audio as well as 3D avatar representation for every user. The platform was integrated into a real-world 5G network to test its capabilities as well as validating the ability of the infrastructure to handle demanding uplink bitrate and computation-intensive XR applications. Both network- and application-level experiments were conducted to measure key performance metrics, including latency, packet loss, and throughput. The results demonstrate the feasibility of the proposed platform and its ability to support real-time communication in collaborative XR scenarios.