Dynamic rules for routing Galaxy entities to destinations
TotalPerspectiveVortex (TPV) is a plugin for the Galaxy application that can route entities (Tools, Users, Roles) to appropriate destinations with appropriate resource alloations (cores, gpus, memory), based on a configurable yaml file. For example, it could allocate 8 cores and 32GB of RAM to a bwa-mem job, and route it to a Slurm cluster, while allocating 2 cores and 4GB of RAM to an upload job, and route it to a local runner. These rules can also be shared community-wide, imported at runtime by any Galaxy deployment, and overridden locally when necessary.
How it works
TPV can be plugged into Galaxy via job_conf.yml. TPVs configuration file specifies how entities
(tools, users, roles) should be allocated resources (cores, gpus, memory) and in complex environments
with multiple job destinations, where to map the resulting jobs to (through a flexible
tagging system). Destinations can have arbitrary scheduling tags defined, and each entity can express a
preference or aversion to specific scheduling tags. This tagging affects how jobs are routed to
destinations. In addition, admins can also plugin arbitrary python based rules for making more complex
decisions, as well as custom ranking functions for choosing between matching destinations.