Diamonds are forever, files are not
Surendar Chandra, Ashish Gehani, Xuwen Yu · File and Storage Technologies · 2007
This work was motivated by the observation that not all objects are equally important. In many scenarios, the importance also wanes with time. We leveraged this importance notions to reduce the management overhead for less important objects. In our system, all objects were explicitly annotated by the content creators using a temporal importance metric. The storage system used these hints to manage the reliability (by reducing the replication levels for less important objects), fidelity (by transcoding objects), performance (by moving more important objects to faster storage locations), security (by encrypting important objects with stronger encryption) and persistence (by reducing the persistence guarantees for less important objects). Our initial focus was on policies for managing the object persistence. Our experiments were motivated by a university wide lecture video capture and storage system. Multimedia objects are large; depending on the quality used, compressed media objects can consume storage at the rate of multiple gigabytes per hour. Multimedia objects can eventually fill even the largest storage. Users were forced to either keep adding more storage on a regular basis or manually delete some objects. Keeping track of objects for deletion is cumbersome; many objects are orphaned and much of the storage remained under utilized. The difficulty was that traditional storage systems considered that all objects were equally important. When the storage was full, the storage system denied requests to store newer objects even though old objects that were already stored in the system were potentially less important than the incoming object. Storage systems relied upon user intervention to reclaim the space consumed by less important objects. We relegated object reclamation policies to the storage system itself. This delegation notions is not new; Roscoe et. al. [1] described a system which evicted objects in a FIFO basis; the persistence achieved depends on the future object arrival rates. Zadok et. al. [2] defined two classes of objects wherein objects in the elastic quota class can be reclaimed as necessary. In their position paper, Douglis et. al. [3] introduced the notions of retention value of objects. Our analysis shows that, FIFO cache like replacement policies are agnostic towards object impor