TY - GEN
T1 - Airavata Metascheduler
T2 - 2023 Practice and Experience in Advanced Research Computing, PEARC 2023
AU - Ranawaka, Isuru
AU - Abeysinghe, Eroma
AU - Wannipurage, Dimuthu
AU - De Silva, Dinuka
AU - Brookes, Emre
AU - Marru, Suresh
AU - Christie, Marcus
AU - Pamidighantam, Sudhakar
AU - Pierce, Marlon
N1 - Publisher Copyright:
© 2023 ACM.
PY - 2023/7/23
Y1 - 2023/7/23
N2 - Software-as-a-service science gateways provide user interfaces and middleware for accessing scientific software deployed on remote high-performance computing resources and clusters. Selecting the resource to use for a particular job submission may be left to the user, who may need more information to make good choices when selecting from multiple options. To address this problem, we have designed and developed an extensible, scalable metascheduling system that can provide automated scheduling capabilities based on resource availability and other characteristics. We develop a system model based on queuing theory to guide our implementation and provide a basis for analysis. In particular, we derive an efficiency metric from these considerations. We implement the metascheduling system within the open-source Apache Airavata framework for science gateways as a supplemental service for guiding the job submission capabilities. We measure efficiency in representative scenarios, observing efficiencies of greater than 70% even in scenarios with high input rates and low job acceptance rates.
AB - Software-as-a-service science gateways provide user interfaces and middleware for accessing scientific software deployed on remote high-performance computing resources and clusters. Selecting the resource to use for a particular job submission may be left to the user, who may need more information to make good choices when selecting from multiple options. To address this problem, we have designed and developed an extensible, scalable metascheduling system that can provide automated scheduling capabilities based on resource availability and other characteristics. We develop a system model based on queuing theory to guide our implementation and provide a basis for analysis. In particular, we derive an efficiency metric from these considerations. We implement the metascheduling system within the open-source Apache Airavata framework for science gateways as a supplemental service for guiding the job submission capabilities. We measure efficiency in representative scenarios, observing efficiencies of greater than 70% even in scenarios with high input rates and low job acceptance rates.
KW - cyberinfrastructure
KW - metascheduling
KW - open source software
KW - queueing analysis
KW - science gateways
UR - http://www.scopus.com/inward/record.url?scp=85176238912&partnerID=8YFLogxK
UR - https://www.mendeley.com/catalogue/99402002-53c3-3086-af4f-560c300e10ee/
U2 - 10.1145/3569951.3593605
DO - 10.1145/3569951.3593605
M3 - Conference contribution
AN - SCOPUS:85176238912
SN - 9781450399852
T3 - PEARC 2023 - Computing for the common good: Practice and Experience in Advanced Research Computing
SP - 35
EP - 42
BT - PEARC 2023 - Computing for the common good: Practice and Experience in Advanced Research Computing
PB - Association for Computing Machinery, Inc
Y2 - 23 July 2023 through 27 July 2023
ER -