Data broker
Lars Schneidenbach, Bruce D’Amora, Claudia Misale, Carlos H. Ã. Costa, Sara I. L. Kokkila-Schumacher, Thomas E. Ward · Proceedings of the International Symposium on Memory Systems · 2019
Complex problems can often only be solved with a workflow of different applications whose progress depends also on the need of shared data. There are different challenges that developers must overcome when sharing data in a workflow; each application may have very different requirements in how they access and consume shared data. For example, data access can be online or offline, data sizes and types may vary as well as the frequency they are consumed and produced by applications in the workflow. Moreover, producers and consumers may not run at the same time scale, thus introducing possible latency. Also, the deployment system itself plays an important role, especially when considering low latency read/write operations and reliability in case of failures.