ICENI Dataflow and Workflow: Composition and Scheduling in Space and Time
Anthony Mayer, Steve McGough, Nathalie Furmento, William Lee, Steven J. Newhouse · 2003
With the prevalence of component based and service oriented architectures used to support e-Science activities, we examine different views of application composition supported within these systems, which tend to be spatial composition in the former case, and temporal composition (workflow) in the latter. We consider the advantages of each view; spatial composition enables, dynamic programming, while temporal composition provides information useful for performance analysis. ICENI uses a spatial composition view to allow maximum flexibility, but provides an inferred temporal composition to support scheduling optimisation. We describe the graph based language used to annotate component behaviours, and discuss optimisations derived from estimating execution time- with or without loops- and resource sharing. Two examples drawn from e-Science pilot projects are used to illustrate these techniques.