TotalPerspectiveVortex (TPV) provides an installable set of dynamic rules for the Galaxy application that can route entities (Tools, Users, Roles) to appropriate destinations based on a configurable yaml file. The aim of TPV is to build on and unify previous efforts, such as Dynamic Tool Destinations, the Job Router and Sorting Hat, into a configurable set of rules that that can be extended arbitrarily with custom Python logic.

How it works

TPV provides a dynamic rule that can be plugged into Galaxy via job_conf.xml. The dynamic rule will also have an associated configuration file, that maps entities (tools, users, roles) to specific destination through a flexible tagging system. Destinations can have arbitrary tags defined, and each entity can express a preference or aversion to specific tags. Based on this tagging, jobs are routed to the most appropriate destination. In addition, admins can also plugin arbitrary python based rules for making more complex decisions, as well as custom ranking functions for choosing between matching destinations. For example, a ranking function could query influx metrics to determine the least loaded destination, and route jobs there, providing a basic form of “metascheduling” functionality.

Getting Started

  1. pip install total-perspective-vortex into Galaxy’s python virtual environment

  2. Configure Galaxy to use TPV’s dynamic destination rule

  3. Create the TPV job mapping yaml file, indicating job routing preferences

  4. Submit jobs as usual

Indices and tables