Hosting dynamic data in the cloud with Isis2 and the Ida DHT
Ken Birman, Heesung Sohn · 2013
The big-data community generally favors a two stage methodology whereby data is first collected, then uploaded for analysis using tools like MapReduce. During analysis the data won't change; this simplifies fault-tolerance and makes it worthwhile to cache intermediary results. In contrast, when it is necessary to capture data continuously and query it on the fly, cloud storage and access technologies must be reexamined. Isis2 aims at such scenarios, offering a base set of mechanisms that replicate data and perform computation with strong consistency and other assurance properties, then layering higher level abstractions over this core. Here we a focus on a subsystem called the Isis2 interactive data analysis infrastructure: Ida. Ida is a strongly-consistent distributed key-value store on which surprisingly complex computational tasks are feasible.