Scheduling of Data Transcription in Periodically Connected Databases
Avigdor Gal, Jonathan Eckstein, Zachary G. Stoumbos · Stochastic Analysis and Applications · 2003
Contemporary data management in applications such as pervasive systems, Web‐based supply chain management, data warehouses, and Web crawlers involve the periodic transcription of data onto secondary devices in a networked environment. In this paper, we focus on the scheduling of periodic data transcription in append‐only environments, such as e‐mail inboxes, newsgroups, technical support bulletin boards, or procurement requests. If the client connects to the server too frequently, the client will nearly always have up‐to‐date information, but the usage of network resources may be excessive. Conversely, very infrequent connections will conserve network resources, but the client's data may often be significantly out of date, which may also be costly (in terms of lost opportunities, for example). Thus, the best transcription policies should make on optimal trade‐off between these costs. Our approach to evaluating this trade‐off is to use modeling techniques from the field of stochastic processes. The paper presents a general model for data insertions on the server side, using compound nonhomogeneous Poisson processes, and compares several transcription policies in terms of both transcription cost and obsolescence cost. The comparisons use a validation data set from a real data feed, and our models were calibrated using a separate training set from the same feed. We find that transcription policies based on a nonhomogeneous Poisson arrival model often outperform simpler policies.