The mosaic camera: synthesizing a large-format camera from many smaller cameras
Gene Howard Golub, Donald Tanguay · 2006
New video applications are becoming possible with the advent of several enabling technologies: multicamera capture, increased PC bus bandwidth, multicore processors, and advanced graphics cards. This thesis presents one such application, the camera---compositing images from many cameras into a single high-resolution, panoramic video. We describe novel methods for building a coherent system of homographies for mosaic construction. Unlike traditional point-based methods, we use shared observations of lines to constrain the placement of images. This has two significant advantages: (1) mosaic resolution is maximized by reducing or eliminating overlap in the original images, and (2) lines provide more global constraints by simultaneously relating both adjacent and distant cameras. In fact, two non-overlapping cameras will not share any scene points; however, if they see different parts of the same line they have common information that constrains their spatial alignment. Our approach involves both closed-form and iterative solutions. The closed-form solution determines a common system of homographies from a set of pair-wise homographies, with novelty in its exploitation of cyclic image relationships. Our iterative global bundle adjustment formulation improves upon the closed-form solution, attaining high-quality results. We introduce 3 error evaluation measures for assessing the quality of a mosaic and show the performance of our methods in simulations. We also describe many of the practical aspects of calibrating a mosaic camera. In order to fully automate the calibration process, we use a digital projector as a calibration instrument. We must overcome low dynamic range and measure image properties with good accuracy. Because these image measurements are a significant percentage of the calibration time, we also describe methods for reducing the number of measurements needed. Finally, we present our implementation and results. We describe how we implement the mosaic camera using a commercially-available multicamera system, a software architecture for media processing, and powerful graphics cards. We show results of 3 different mosaic cameras that produce up to 7 megapixel video at 30 Hz on a single PC. We also demonstrate an interactive desktop application in which a user can digitally manipulate a nonuniform-resolution virtual camera with pan, tilt, zoom, and roll motions.