BioBlend — BioBlend 0.5.4-dev documentation

# BioBlend

## About

BioBlend is a Python (2.6 or 2.7) library for interacting with CloudMan and Galaxy‘s API.

Conceptually, it makes it possible to script and automate the process of cloud infrastructure provisioning and scaling via CloudMan, and running of analyses via Galaxy. In reality, it makes it possible to do things like this:

- Create a CloudMan compute cluster, via an API and directly from your local machine:

      from bioblend.cloudman import CloudManConfig
      from bioblend.cloudman import CloudManInstance
      cfg = CloudManConfig('<your cloud access key>', '<your cloud secret key>', 'My CloudMan',  'ami-<ID>', 'm1.small', '<password>')
      cmi = CloudManInstance.launch_instance(cfg)
      cmi.get_status()

- Reconnect to an existing CloudMan instance and manipulate it:

      from bioblend.cloudman import CloudManInstance
      cmi = CloudManInstance("<instance IP>", "<password>")
      cmi.add_nodes(3)
      cluster_status = cmi.get_status()
      cmi.remove_nodes(2)

- Interact with Galaxy via a straightforward API:

      from bioblend.galaxy import GalaxyInstance
      gi = GalaxyInstance('<Galaxy IP>', key='your API key')
      libs = gi.libraries.get_libraries()
      gi.workflows.show_workflow('workflow ID')
      gi.workflows.run_workflow('workflow ID', input_dataset_map)

- Interact with Galaxy via an object-oriented API:

      from bioblend.galaxy.objects import GalaxyInstance
      gi = GalaxyInstance("URL", "API_KEY")
      wf = gi.workflows.list()[0]
      hist = gi.histories.list()[0]
      inputs = hist.get_datasets()[:2]
      input_map = dict(zip(wf.input_labels, inputs))
      params = {"Paste1": {"delimiter": "U"}}
      wf.run(input_map, "wf_output", params=params)

Note

Although this library allows you to blend these two services into a cohesive unit, the library itself can be used with either service irrespective of the other. For example, you can use it to just manipulate CloudMan clusters or to script the interactions with an instance of Galaxy running on your laptop.

## Installation

Stable releases of BioBlend are best installed via `pip` or `easy_install` from PyPI using something like:

    $ pip install bioblend

Alternatively, you may install the most current source code from our Git repository, or fork the project on Github. To install from source, do the following:

    # Clone the repository to a local directory
    $ git clone https://github.com/galaxyproject/bioblend.git
    # Install the library
    $ cd bioblend
    $ python setup.py install

After installing the library, you will be able to simply import it into your Python environment with `import bioblend`. For details on the available functionality, see the API documentation.

BioBlend requires a number of Python libraries. These libraries are installed automatically when BioBlend itself is installed, regardless whether it is installed via PyPi or by running `python setup.py install` command. The current list of required libraries is always available from setup.py in the source code repository and it is also replicated here: `requests>=1.1.0, poster, boto, pyyaml`.

If you will also run tests locally, the following libraries are also required: `nose, mock`. To install those, run:

    $ python setup.py test

## Usage

To get started using BioBlend, install the library as described above. Once the library becomes available on the given system, it can be developed against. The developed scripts do not need to reside in any particular location on the system.

It is probably best to take a look at the example scripts in `docs/examples` source directory and browse the API documentation. Beyond that, it’s up to your creativity :).

## Development

Anyone interested in contributing or tweaking the library is more then welcome to do so. To start, simply fork the Git repository on Github and start playing with it. Then, issue pull requests.

## API Documentation

BioBlend’s API focuses around and matches the services it wraps. Thus, there are two top-level sets of APIs, each corresponding to a separate service and a corresponding step in the automation process. Note that each of the service APIs can be used completely independently of one another.

Effort has been made to keep the structure and naming of those API’s consistent across the library but because they do bridge different services, some discrepancies may exist. Feel free to point those out and/or provide fixes.

For Galaxy, an alternative object-oriented API is also available. This API provides an explicit modeling of server-side Galaxy instances and their relationships, providing higher-level methods to perform operations such as retrieving all datasets for a given history, etc. Note that, at the moment, the oo API is still incomplete, providing access to a more restricted set of Galaxy modules with respect to the standard one.

### CloudMan API

API used to manipulate the instantiated infrastructure. For example, scale the size of the compute cluster, get infrastructure status, get service status.

- API documentation for interacting with CloudMan
  - CloudManLauncher
  - CloudManInstance
- Usage documentation
  - Setting up custom cloud properties
  - Launching a new cluster instance
  - Manipulating an existing cluster

### Galaxy API

API used to manipulate genomic analyses within Galaxy, including data management and workflow execution.

- API documentation for interacting with Galaxy
  - GalaxyInstance
  - Datasets
  - Datatypes
  - Folders
  - Forms
  - FTP files
  - Genomes
  - Groups
  - Histories
  - Jobs
  - Libraries
  - Quotas
  - Roles
  - Tools
  - Tool data tables
  - ToolShed
  - Users
  - Visual
  - Workflows
- Object-oriented Galaxy API
  - Client
  - Wrappers
- Usage documentation
  - Connect to a Galaxy server
  - View Histories and Datasets
  - View Data Libraries
  - Upload files to a Data Library
  - View Workflows
  - Export or import a Workflow
  - Run a Workflow
  - View Users

### Toolshed API

API used to interact with the Galaxy Toolshed, including repository management.

- API documentation for interacting with the Galaxy Toolshed
  - ToolShedInstance
  - Repositories

## Configuration

BioBlend allows library-wide configuration to be set in external files. These configuration files can be used to specify access keys, for example.

- Configuration documents for BioBlend

## Testing

If you’d like to do more than just a mock test, you’ll want to point BioBlend to an instance of Galaxy. Do so by exporting the following two variables:

    $ export BIOBLEND_GALAXY_URL=http://127.0.0.1:8080
    $ export BIOBLEND_GALAXY_API_KEY=<API key>

The unit tests, stored in the `tests` folder, can be run using nose. From the project root:

    $ nosetests

## Getting help

If you’ve run into issues, found a bug, or can’t seem to find an answer to your question regarding the use and functionality of BioBlend, please use Github Issues page to ask your question.

## Related documentation

Links to other documentation and libraries relevant to this library:

> - Galaxy API documentation
> - Blend4j: Galaxy API wrapper for Java
> - clj-blend: Galaxy API wrapper for Clojure

## Indices and tables

- Index
- Module Index
- Search Page