Installation

Basic steps

  1. pip install total-perspective-vortex into Galaxy’s python virtual environment

  2. Create the TPV job mapping yaml file, specifying resource allocation and job routing preferences

  3. Configure Galaxy’s job_conf.yml to use TPV

  4. Submit jobs as usual

Configuring Galaxy

  1. First install TPV into your Galaxy virtual environment.

    TPV is a conditional dependency of Galaxy since Galaxy 22.05. If TPV is enabled in your Galaxy job configuration, it will automatically be installed into Galaxy’s virtualenv. Otherwise, or if you wish to upgrade to a newer version of TPV, you can use the process below to install manually:

    cd <galaxy_home>
source .venv/bin/activate
pip install --upgrade total-perspective-vortex

  2. Edit your job_conf.yml in the <galaxy_home>/config folder and add the highlighted sections to it.

    You can refer to a local file for the tpv_config_files setting, or alternatively, provider a link to a remote url.

     1runners:
 2  local:
 3    load: galaxy.jobs.runners.local:LocalJobRunner
 4    workers: 4
 5  drmaa:
 6    load: galaxy.jobs.runners.drmaa:DRMAAJobRunner
 7  k8s:
 8    load: galaxy.jobs.runners.kubernetes:KubernetesJobRunner
 9
10handling:
11  assign:
12    - db-skip-locked
13
14execution:
15  default: tpv_dispatcher
16  environments:
17    tpv_dispatcher:
18      runner: dynamic
19      type: python
20      function: map_tool_to_destination
21      rules_module: tpv.rules
22      tpv_config_files:
23        - https://gxy.io/tpv/db.yml
24        - config/tpv_rules_local.yml
25    local:
26      runner: local
27    k8s_environment:
28      runner: k8s
29      docker_enabled: true

  3. Add your own custom rules to your local tpv_config_file, following instructions in the next section.

Combining multiple remote and local configs

TPV allows rules to be loaded from remote or local sources.

1tpv_dispatcher:
2 runner: dynamic
3 type: python
4 function: map_tool_to_destination
5 rules_module: tpv.rules
6 tpv_config_files:
7   - https://gxy.io/tpv/db.yml
8   - config/tpv_rules_australia.yml

The config files listed first are overridden by config files listed later. The normal rules of inheritance apply. This allows a central database of common rules to be maintained, with individual, site-specific overrides.

Standalone Installation

If you wish to install TPV outside of Galaxy’s virtualenv (e.g. to use the tpv lint command locally or in a CI/CD pipeline), use the [cli] pip requirement specifier to make sure the necessary Galaxy dependency packages are also installed. This should not be used in the Galaxy virtualenv:

$ pip install 'total-perspective-vortex[cli]'