Algorithms for Dispersed Processing
Josef Spillner, Alexander Schill · 2014
Highly scalable computing environments demand a parallelisation and distribution of processing tasks. Consequently, the data being processed is redundantly distributed to benefit from data locality characteristics, but also to increase safety, privacy and security objectives. Such distributed processing is coordinated by message passing or map-reduce programming styles. For partially replicated and dispersed data, however, the distributed processing poses new challenges because the required input data elements are not wholly available anymore to the processing tasks. Novel and adjusted processing algorithms which work under restricted assumptions thus become an important part of distributed infrastructures. We review, propose and analyse algorithms which align with split and dispersed data structures. Subsequently, we contribute and evaluate our implementations thereof in order to assess possible future applications on top of dispersed storage and multipath transmission of data.