<!--

#################################################
### THIS FILE WAS AUTOGENERATED! DO NOT EDIT! ###
#################################################
# file to edit: index.ipynb
# command to build the docs after a change: nbdev_build_docs

-->

# NBX: Notebook Experiments for OpenMind

> This module enables you to quickly convert your jupyter notebook into a bundle of files that can be run on BCS'*OpenMind*. 


# Getting started

## Prerequisites
#### Install the package:

- `pip install nbx`

#### Get a singulartiy image
You'll need an image that has the package installed (there are ways around that, but I am keeping it simple at the moment). Here's the one I use most of the time:

```
module load openmind/singularity/3.2.0
export SINGULARITY_CACHEDIR="/om2/user/{your_user_name}/.singularity"
singularity build pytorch.simg docker://mklukas/pytorch
```

#### Environment variables
For the modules to work you have to set the environment variables `om`, `omx`, `omsimg`, and `omid`:  

- **om**: your login to *OpenMind*. 
    - You need to enable logging into *OpenMind* using public key authentication. That means the command `ssh $om` should log you in whithout asking for a password. (googling for "ssh public key authentication" will provide you with a recipe like [this](https://kb.iu.edu/d/aews))
- **omx**: path to the folder where *nbx* bundles are stored. This path will automatically be added to your python path. Any modules that are not part of your bundles `src/` folder or are included in your singularity container should go here.
- **omsimg**: path to the folder containing your singularity images
- **omid**: your Open Mind username 

Mac users can adapt and copy the following lines to their `.bash_profile` file

```
export om={your_user_name}@openmind7.mit.edu
export omid={your_user_name}
export omx=/om2/user/{your_user_name}/nbx-experiments
export omsimg=/om2/user/{your_user_name}/simg
```



## Usage

- Put your python scripts that you wrote for this experiment in a folder called `src`. The folder will be copied to the bundle so the scripts are available on *OpenMind* as well.
- **#nbx**: Each cell that contains a `#nbx` tag in its **first** line will be considered part of the experiment.
- **#xarg**: Putting `#xarg` above a variable declaration makes this variable *explicit*, it will become an argument of the experiment function. Any iterable to the right of the variable declaration, **separated by a semicolon**, will be considered the domain that will be swept during the parameter sweep.
- Each nbx-experiment has to declare the variables `task_id` and `results_dir`. The *task id* will be set by the *wrapper* script and enumerates the configurations of the parameter space. The latter variable will also be set by the *wrapper* script, it will be replaced by the folder automatically created for a specific parameter configuration. 

# Example
<div class="codecell" markdown="1">
<div class="input_area" markdown="1">

```python
%matplotlib inline
%load_ext autoreload
%autoreload 2
```

</div>
<div class="output_area" markdown="1">

    The autoreload extension is already loaded. To reload it, use:
      %reload_ext autoreload


</div>

</div>
<div class="codecell" markdown="1">
<div class="input_area" markdown="1">

```python
#nbx
#xarg
task_id = 0
#xarg
results_dir = "./"

#xarg
x=0;

#xarg
y=0; [0,1,2,3,4]

z=0;
```

</div>

</div>
<div class="codecell" markdown="1">
<div class="input_area" markdown="1">

```python
#nbx
print("my results:", x, y, z)
```

</div>
<div class="output_area" markdown="1">

    my results: 0 0 0


</div>

</div>
<div class="codecell" markdown="1">
<div class="input_area" markdown="1">

```python
from nbx.om import NbxBundle

bundle = NbxBundle(nbname="index.ipynb", # the name of the notebook
          name="test",                   # name of the bundle
          linting=False,                  
          time=[2,0],                     
          ntasks=4,                      
          step=5,                        
          mail_user="me@somewhere.com",
          simg="pytorch.simg")
```

</div>
<div class="output_area" markdown="1">

    [('task_id', '0', ''), ('results_dir', '"./"', ''), ('x', '0', ''), ('y', '0', '[0,1,2,3,4]')]
    /Users/mirko/Workspace/nbx/nbx/templates/experiment.tpl
    /Users/mirko/Workspace/nbx/nbx/templates/wrapper.tpl
    /Users/mirko/Workspace/nbx/nbx/templates/run.tpl
    
    ** nbx bundle created **
    Path:
        test_nbx
    
    Source nb:
        index.ipynb
    
    Parameters (#configs 5):
        * y = [0,1,2,3,4]
          task_id = 0
          results_dir = "./"
          x = 0
    
    Instructions:
        Copy to remote, run the bash script, and pull the results
        - `bundle.push()` or `scp -r test_nbx $om:$omx`
        - `bundle.run()` or `ssh $om sbatch -D $omx/test_nbx $omx/test_nbx/run.sh`
        - `bundle.pull_results()` or `scp -r $om:$omx/test_nbx/results ./results`


</div>

</div>
