Dynamic rules for routing Galaxy entities to destinations

TotalPerspectiveVortex (TPV) is a plugin for the Galaxy application that can route entities (Tools, Users, Roles) to appropriate destinations with appropriate resource alloations (cores, gpus, memory), based on a configurable yaml file. For example, it could allocate 8 cores and 32GB of RAM to a bwa-mem job, and route it to a Slurm cluster, while allocating 2 cores and 4GB of RAM to an upload job, and route it to a local runner. These rules can also be shared community-wide, imported at runtime by any Galaxy deployment, and overridden locally when necessary.

How it works

TPV can be plugged into Galaxy via job_conf.yml. TPVs configuration file specifies how entities (tools, users, roles) should be allocated resources (cores, gpus, memory) and in complex environments with multiple job destinations, where to map the resulting jobs to (through a flexible tagging system). Destinations can have arbitrary scheduling tags defined, and each entity can express a preference or aversion to specific scheduling tags. This tagging affects how jobs are routed to destinations. In addition, admins can also plugin arbitrary python based rules for making more complex decisions, as well as custom ranking functions for choosing between matching destinations.

Shared database

A shared database of TPV rules are maintained in: https://github.com/galaxyproject/tpv-shared-database/ These rules are based on typical settings used in the usegalaxy.* federation, which you can override based on local resource availability.

Indices and tables