Learning a model from spatially disjoint data
Lawrence Hall, Divya Bhadoria, Kevin W. Bowyer · 2005
Some large-scale simulations are distributed over thousands of processors and generate terabytes of data. The output may take weeks or months to debug and explore. Therefore, learning a model that allows users to quickly focus on interesting events would be a great timesaver. Training data will not fit in one physical memory and the most natural splitting of the data into tractable size subsets would be along lines of disk farms, causing data for simulated objects to be split across training sets. In general, the training sets contain very few interesting examples. A k nearest centroids approach was developed to classify unseen data. ROC analysis on a set of face images (partitioned spatially) indicates that this is a promising approach for the larger problem.