Introduction Dataow Collection and Scheduling Policies Simulation Results WaFS Prototype Concluding Remarks
Dataow Collection and Scheduling Policies
Yang Wang
Department of Computing Science University of Alberta, Canada yangwang@cs.ualberta.ca
27th July 2008
Yang Wang
Transparent Dataow Detection and Use in Workow Scheduling:
Introduction Dataow Collection and Scheduling Policies Simulation Results WaFS Prototype Concluding Remarks
Outline
1
Introduction Workow-based Workloads in HPC Goal and Challenges Our Contributions Dataow Collection and Scheduling Policies Motivation Dataow Collection: WaFS Scheduler Dataow-based Scheduling Policies Simulation Results WaFS Prototype WaFS Architecture Batch Scheduler and Versioned Namespace Manager Monitor: Dataow Collection A Running Example Concluding Remarks
2
3 4
5
Yang Wang
Transparent Dataow Detection and Use in Workow Scheduling:
Introduction Dataow Collection and Scheduling Policies Simulation Results WaFS Prototype Concluding Remarks
Workow-based Workloads in HPC Goal and Challenges Our Contributions
Workow-based Workloads in HPC
A computational workow consists of a set of control/data dependent jobs,represented as a directed acyclic graph (DAG), to carry out a well-dened scientic computing process. Control-ow: A sequencing dependency between any two jobs in a workow (i.e., Job A must nish before Job B is started). Dataow: A producer-consumer data relationship between two (or more) jobs where, a job cannot start until its required input data is ready.
Yang Wang
Transparent Dataow Detection and Use in Workow Scheduling:
Introduction Dataow Collection and Scheduling Policies Simulation Results WaFS Prototype Concluding Remarks
Workow-based Workloads in HPC Goal and Challenges Our Contributions
Goal and Challenges
Goal To maximize the performance of multiple instances of the same scientic workow (i.e., workow-based scientic workload) Challenges : Given the ease with which parameter-based studies can generate jobs and workow instances, storage can often be a constraining resource, and lename conict is thus a potential concern. However, in most current HPC systems, the batch scheduler is control-ow-based and thus not efcient to address the lename conict problem.

下一页