Scaling Filename Queries in a Large-Scale Distributed File System
Jonathan Ledlie, Laura Serban, Dafina Toncheva · Digital Access to Scholarship at Harvard (DASH) (Harvard University) · 2002
The grouping topology we have developed divides nnodes into log(n) groups, each of which has a repre-sentative node that aggregates a composite filter for the group. All nodes not in that group use this low-precisionfilter to weed out whole collections of nodes by probing these filters, only sending a search to be proxied bya member of the group if the probe of the group filter returns positively. Proxied searches are then carried outwithin a group, where more precise (more bits per file) filters are kept and exchanged between the nlog(n) nodesin a group. Experimental results show that both bandwidth/search and time/search are improved with this novelgrouping topology.