ARCHIE: A User-Focused Framework for Testing Augmented Reality Applications in the Wild
Sarah M. Lehman, Haibin Ling, Chiu C. Tan · 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) · 2020
In this paper, we present ARCHIE, a framework for testing augmented reality applications in the wild. ARCHIE collects user feedback and system state data in situ to help developers identify and debug issues important to testers. It also supports testing of multiple application versions (called "profiles") in a single evaluation session, prioritizing those versions which the tester finds more appealing. To evaluate ARCHIE, we implemented four distinct test case applications and used these applications to examine the performance overhead and context switching cost of incorporating our framework into a pre-existing code base. With these, we demonstrate that ARCHIE provides no significant overhead for AR applications, and introduces at most 2% processing overhead when switching among large groups of testable profiles.