Interact with Mantle Pipeline runs

Introduction

Each time you run a Mantle Pipeline, a pipeline run object is generated. A pipeline run contains links to its input entities and parameters and its output entities.

Getting a pipeline run by unique ID

Get a pipeline run object by its unique ID

import mantlebio

mantle = mantlebio.MantleClient()

# Get a pipeline run
pipeline_run = mantle.pipeline_run.get(<'pipeline_run_id'>)

This returns a Python object representing the run.

Getting single entity outputs of a pipeline run

All outputs of a pipeline run expressed as key-value pairs. You can get a single entity output of a pipeline run using its key.

Get single entity output

import mantlebio

mantle = mantlebio.MantleClient()

# Get a pipeline run
pipeline_run = mantle.pipeline_run.get('pipeline_run_id')

# Get a entity output of the run using the key
pipeline_run.get_output('output_entity_key')

This returns a Python object representing the data entity.

Getting inputs of a pipeline run

You may need to access the inputs of a pipeline run to incorporate them into your downstream analysis.

All inputs of a pipeline run expressed as key-value pairs.

Getting a single entity input

If a pipeline takes a single entity input for a key, you can get a Python object representing the entity used in a specific run using the key:

Get single input entity

import mantlebio

mantle = mantlebio.MantleClient()

# Get a pipeline run
pipeline_run = mantle.pipeline_run.get('pipeline_run_id')

# Get a entity input of the run using the key
pipeline_run.get_input_dataset('input_entity_key')

This returns a Python object representing the data entity.

For example, the Mantle nf-core rnaseq pipeline takes a single entity for the reference genome:

Get reference entity for bulk-rnaseq run

import mantlebio

mantle = mantlebio.MantleClient()

# Get a bulk-rnaseq pipeline run
pipeline_run = mantle.pipeline_run.get('bulk-rnaseq_pipeline_run_id')

# Get the reference genome entity of the run
pipeline_run.get_input_dataset('reference')

Getting a list of entity inputs

If a pipeline can take multiple entity inputs for a key, you can get a list of the entities used in a specific run using the key:

Get a list of input entities

import mantlebio

mantle = mantlebio.MantleClient()

# Get a pipeline run
pipeline_run = mantle.pipeline_run.get('pipeline_run_id')

# Get a entity input of the run using the key
pipeline_run.get_input_dataset_list('input_entity_key')

This returns a Python list of entity objects.

For example, the Mantle nf-core rnaseq pipeline can take multiple rnaseq_fastq entities for the fastqs input:

Get fastq entities for bulk-rnaseq run

import mantlebio

mantle = mantlebio.MantleClient()

# Get a bulk-rnaseq pipeline run
pipeline_run = mantle.pipeline_run.get('bulk-rnaseq_pipeline_run_id')

# Get a list of the rnaseq_fastq inputs
pipeline_run.get_input_dataset_list('fastqs')

Advanced

Pipeline development

To add a custom pipeline you have developed to Mantle, please contact us.

Adding outputs to pipeline runs

Adding a entity output to a pipeline run

Add entity output

import mantlebio

mantle = mantlebio.MantleClient()

# Create the entity, e.g.
entity = mantle.dataset.create(
    name="4XP1",
    local=False,
    properties={
        "description": "X-ray structure of Drosophila dopamine transporter bound to neurotransmitter dopamine",
        "resolution": 2.5,
        "r_value": 0.2,
    }
)

# Get the pipeline run you want to add outputs to
pipeline_run = mantle.pipeline_run.get('pipeline-run-id')

# Add dataset output to the pipeline run
pipeline_run.add_dataset_output('<key_for_output_entity>', entity)

Adding a file output to a pipeline run

File outputs can be downloaded directly from the pipeline run page in the Mantle Pipelines UI. Some files (such as HTMLs, CSVs, PDBs, PNGs, and JPGs) can be previewed on the pipeline run page. This can be useful for displaying reports to pipeline users.

Add file output

import mantlebio

mantle = mantlebio.MantleClient()

# Get the pipeline run you want to add outputs to
pipeline_run = mantle.pipeline_run.get('pipeline-run-id')

# Add file output to a pipeline run
pipeline_run.add_file_output('<key_for_output_file>','local/file/path/to/file')