Airavata Metascheduler: A Reliable, Fault Tolerant, and Resource-Aware Job Scheduling Service

Isuru Ranawaka, Eroma Abeysinghe, Dimuthu Wannipurage, Dinuka De Silva, Emre Brookes, Suresh Marru, Marcus Christie, Sudhakar Pamidighantam, Marlon Pierce

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Software-as-a-service science gateways provide user interfaces and middleware for accessing scientific software deployed on remote high-performance computing resources and clusters. Selecting the resource to use for a particular job submission may be left to the user, who may need more information to make good choices when selecting from multiple options. To address this problem, we have designed and developed an extensible, scalable metascheduling system that can provide automated scheduling capabilities based on resource availability and other characteristics. We develop a system model based on queuing theory to guide our implementation and provide a basis for analysis. In particular, we derive an efficiency metric from these considerations. We implement the metascheduling system within the open-source Apache Airavata framework for science gateways as a supplemental service for guiding the job submission capabilities. We measure efficiency in representative scenarios, observing efficiencies of greater than 70% even in scenarios with high input rates and low job acceptance rates.

Original languageEnglish
Title of host publicationPEARC 2023 - Computing for the common good: Practice and Experience in Advanced Research Computing
Subtitle of host publicationPractice and Experience in Advanced Research Computing
PublisherAssociation for Computing Machinery, Inc
Pages35-42
Number of pages8
ISBN (Electronic)9781450399852
ISBN (Print)9781450399852
DOIs
StatePublished - Jul 23 2023
Event2023 Practice and Experience in Advanced Research Computing, PEARC 2023 - Portland, United States
Duration: Jul 23 2023Jul 27 2023

Publication series

NamePEARC 2023 - Computing for the common good: Practice and Experience in Advanced Research Computing

Conference

Conference2023 Practice and Experience in Advanced Research Computing, PEARC 2023
Country/TerritoryUnited States
CityPortland
Period07/23/2307/27/23

Keywords

  • cyberinfrastructure
  • metascheduling
  • open source software
  • queueing analysis
  • science gateways

Fingerprint

Dive into the research topics of 'Airavata Metascheduler: A Reliable, Fault Tolerant, and Resource-Aware Job Scheduling Service'. Together they form a unique fingerprint.

Cite this