Maintaining Temporal Coherency of Cooperating Dynamic Data Repositories
Shetal Shah Allister, Bernard Vivek Sharma, Krithi Ramamritham, Prashant Shenoyy · 2008
On-line decision making often involves significant amount of time-varying data. Examples of time-varying data include financial information such as stock prices and currency exchange rates, real-time traffic and weather information, and data from industrial process control applications. The coherency requirements associated with time-varying data depends on the nature of the data and user tolerances. This paper examines techniques to efficiently disseminate time-varying data from sources to a set of repositories. A particular focus of our work is to examine how such repositories can cooperate with one another and the source to improve the efficiency of the dissemination process, while meeting user coherency requirements. We consider two key issues: (i) When should the source and/or the repositories push changes of interest to other repositories so as to meet all user-specified coherency requirements? (ii) How should an overlay network of such repositories be organized so as to minimize the overheads of maintaining temporal coherency of all data items stored in the various repositories? We examine these questions in turn, offer a set of alternative solutions to address these questions and experimentally evaluate their performance using real-world traces of dynamically changing data (specifically, stock prices). We show that cooperation helps reduce the systemwide overheads for maintaining coherency across all repositories. However, contrary to intuition, we also show that increasing the degree of cooperation beyond a certain point can, in fact, be detrimental to the overall goals of achieving high fidelity at low overheads. To address this issue, we propose techniques to (i) derive the “optimal” degree of cooperation among repositories, and (ii) derive the logical structure of an overlay network of cooperating repositories so as to maintain temporal coherency of data at low cost.