Visual Perception and Fatigue in AR/VR Head‐Mounted Displays
Yi‐Pai Huang · Information Display · 2019
Yi-Pai Huang Ever since Wheatstone invented the stereoscope in 1838, there has been a steady march toward creating more immersive and realistic images—indeed, it is one of the most attractive topics in the display field. In 1968, a computer scientist developed the world’s first head-mounted display (HMD) at MIT’s Lincoln Laboratory. The device was so heavy and required so many connections that it was attached to a mechanical arm suspended from the ceiling and earned the name “Sword of Damocles.” Fifty years later, thanks to enormous improvements in graphic, computation, and display technologies, wearable augmented and virtual reality (AR/VR) HMDs are now much more practical and are being implemented in a number of different fields, including entertainment, education, and medicine. However, there are still many issues that need to be improved to make the user’s HMD experience more immersive. Examples include widening the available field of view, reducing the so called “screen door” effect of low resolution, and creating better occlusion effects in complex images. The most important one, though, is visual fatigue—and eliminating this effect requires a deeper understanding of the human visual system. Visual fatigue is a more complex problem for AR/VR systems than it is for other display technologies. Vergence-accommodation conflict, which is caused by the conflict between the eye’s perception of the image plane’s focal distance vs the apparent image depth in a stereoscopic scene, has often been discussed as a source of fatigue. Motion sickness, which is induced during the cognitive processing of orientation cues in the visual sensation of body movement, is also a well-known source of fatigue. In this edition of Information Display, we examine the temporal domain effect in human vision. To provide a comfortable visual experience, high frame rate displays, such as the 90 hertz (Hz) OLED, have been widely used in VR HMDs. However, another solution is to use a 120-Hz or higher frame rate LCD with a blinking backlight, which yields better motion picture quality. The article “Temporal Requirements for VR Displays to Create a More Comfortable and Immersive Visual Experience” (page 9) by David M. Hoffman and Grace Lee discusses the relationship between flicker, motion blur, and image brightness as well as frame-to-frame crosstalk. In addition to high frame rate displays, interactive and motion-to-photon latency must also be considered in order to overcome latency fatigue. These factors significantly impact how people perceive the images and are as integral as the optics and the rendering system. One way to improve resolution and optical efficiency in HMDs is to switch from a color-filter subpixel format to a field-sequential color (FSC) format. This allows for three times as many image pixels in each axis and eliminates the light transmission loss in color filters. To make the color subframes appear as one merged color image, designs typically start with subframe rates of 180 Hz or higher. However, FSC displays have a unique side effect known as color breakup (CBU). This effect—which appears as a flash in time when the subframe primary color images are observed individually—also makes users feel uncomfortable and is especially perceivable during rapid eye movement while observing the FSC images. With the article “A Review of Color Breakup Assessment for Field Sequential Color Displays” (page 13), by Zong Qin et al., we examine methods for judging the CBU level, from the color patch to full-color images. The evaluation index also includes visual aspects to match human perception. As an industry, we should continue to address visual fatigue, to broaden AR/VR technology’s applicability and user acceptance. In the meantime, while we may not have a perfect system to prevent fatigue, we can improve our understanding of the fundamental relation between fatigue mechanisms and human perception. Here’s to the possibility of fatigue-free HMDs in the near future. Yi-Pai Huang, Ph.D., is a full professor and the director of the Display Center at National Chiao Tung University in Taiwan. He received the SID Peter Brody Prize in 2017 and became an SID Fellow in 2019. His expertise includes visual and color science, 3D and AR/VR technology, and display optics. Huang can be reached at [email protected].