Speaker
Description
We will present an XR testbed called ILLIXR (Illinois Extended Reality testbed), an open-source end-to-end XR system. ILLIXR supports XR perception, visual, and audio subsystems, consisting of state-of-the-art sensors and components (e.g., visual inertial odometry, scene reconstruction, asynchronous reprojection, and 3D spatial audio encoding and decoding), all orchestrated through a flexible and efficient runtime system. Enabled by Monado, ILLIXR runs XR applications conforming to the OpenXR interface. It runs on Linux PCs and embedded systems (e.g., NVIDIA Jetson), provides the option to offload some components to the cloud (e.g., AWS), and displays images on multiple commercial headsets. It provides extensive telemetry, enabling extensive power, performance, and quality of service measurements and insights on a fully functional XR system.
ILLIXR has led to a consortium with industrial and academic partners with the goal of democratizing XR research, development, and benchmarking. The consortium aims to establish a reference open-source testbed, a standard benchmarking methodology, and a cross-disciplinary R&D community for XR systems. ILLIXR is already being used in a variety of research projects to enable advances in XR hardware, software, systems, and algorithms, with the goal of improving end-to-end user quality of experience. This research includes designing new hardware accelerators that are codesigned with innovations in software algorithms for XR; codesigning 2.5D and 3D packaging technologies (for sensors, compute, and memory) with architectures and algorithms; techniques for offloading XR components from power constrained wearables to edge and cloud servers; compilation technologies; scheduling and runtime system design; quality metrics; and more.
