Metadata-Version: 2.1
Name: nbx
Version: 0.0.5
Summary: This module enables you to quickly convert your jupyter notebook into a bundle of files that can be run on BCS'*OpenMind*
Home-page: https://github.com/mirkoklukas/nbx
Author: Mirko Klukas
Author-email: mirko.klukas@gmail.com
License: Apache Software License 2.0
Description: <!--
        
        #################################################
        ### THIS FILE WAS AUTOGENERATED! DO NOT EDIT! ###
        #################################################
        # file to edit: index.ipynb
        # command to build the docs after a change: nbdev_build_docs
        
        -->
        
        # `nbx.om`: Notebook Experiments for OpenMind
        
        > This module enables you to quickly convert your jupyter notebook into a bundle of files that can be run on BCS'*OpenMind*. 
        
        
        # Getting started
        
        ## Prerequisites
        **1.** Install the package:
        
        - `pip install nbx`
        
        **2.** Get a singulartiy that has the package installed. Here's one you can built yourself:
        
        ```shell
        module load openmind/singularity/3.2.0
        export SINGULARITY_CACHEDIR="/om2/user/{your_user_name}/.singularity"
        singularity build pytorch.simg docker://mklukas/pytorch
        ```
        
        **3.** For the modules to work you have to set the environment variables `om`, `omx`, and `omsimg`:  
        
        - **om**: your login to *OpenMind*. 
            - You need to enable logging into *OpenMind* using public key authentication. That means the command `ssh $om` should log you in whithout asking for a password. (googling for "ssh public key authentication" will provide you with a recipe like [this](https://kb.iu.edu/d/aews))
        - **omx**: path to the folder where *nbx* bundles are stored. This path will automatically be added to your python path. Any modules that are not part of your bundles `src/` folder or are included in your singularity container should go here.
        - **omsimg**: path to the folder containing your singularity images
        
        Mac users can adapt and copy the following lines to their `.bash_profile`
        
        ```shell
        export om={your_user_name}@openmind7.mit.edu
        export omx=/om2/user/{your_user_name}/nbx-experiments
        export omsimg=/om2/user/{your_user_name}/simg
        ```
        
        
        
        ## Usage
        
        - Put your python scripts that you wrote for this experiment in a folder called `src`. The folder will be copied to the bundle so the scripts are available on *OpenMind* as well.
        - **#nbx**: Each cell that contains a `#nbx` tag in its **first** line will be considered part of the experiment.
        - **#xarg**: Putting `#xarg` above a variable declaration makes this variable *explicit*, it will become an argument of the experiment function. Any iterable to the right of the variable declaration, **separated by a semicolon**, will be considered the domain that will be swept during the parameter sweep.
        - Each nbx-experiment has to declare the variables `task_id` and `results_dir`. The *task id* will be set by the *wrapper* script and enumerates the configurations of the parameter space. The latter variable will also be set by the *wrapper* script, it will be replaced by the folder automatically created for a specific parameter configuration. 
        
        
        
        
        
        
        # Example
        <div class="codecell" markdown="1">
        <div class="input_area" markdown="1">
        
        ```python
        %matplotlib inline
        %load_ext autoreload
        %autoreload 2
        ```
        
        </div>
        <div class="output_area" markdown="1">
        
            The autoreload extension is already loaded. To reload it, use:
              %reload_ext autoreload
        
        
        </div>
        
        </div>
        <div class="codecell" markdown="1">
        <div class="input_area" markdown="1">
        
        ```python
        #nbx
        #xarg
        task_id = 0
        #xarg
        results_dir = "./"
        
        #xarg
        x=0;
        
        #xarg
        y=0; [0,1,2,3,4]
        
        z=0;
        ```
        
        </div>
        
        </div>
        <div class="codecell" markdown="1">
        <div class="input_area" markdown="1">
        
        ```python
        #nbx
        print("my results:", x, y, z)
        ```
        
        </div>
        <div class="output_area" markdown="1">
        
            my results: 0 0 0
        
        
        </div>
        
        </div>
        <div class="codecell" markdown="1">
        <div class="input_area" markdown="1">
        
        ```python
        from nbx.om import NbxBundle
        
        bundle = NbxBundle(nbname="index.ipynb", 
                  name="test", 
                  linting=False,
                  time=[2,0], 
                  ntasks=4, 
                  step=5, 
                  simg="mirko-datascience.simg")
        ```
        
        </div>
        <div class="output_area" markdown="1">
        
            [('task_id', '0', ''), ('results_dir', '"./"', ''), ('x', '0', ''), ('y', '0', '[0,1,2,3,4]')]
            /Users/mirko/Workspace/nbx/nbx/templates/experiment.tpl
            /Users/mirko/Workspace/nbx/nbx/templates/wrapper.tpl
            /Users/mirko/Workspace/nbx/nbx/templates/run.tpl
            
            ** nbx bundle created **
            Path:
                test_nbx
            
            Source nb:
                index.ipynb
            
            Parameters (#configs 5):
                * y = [0,1,2,3,4]
                  task_id = 0
                  results_dir = "./"
                  x = 0
            
            Instructions:
                Copy to remote, run the bash script, and pull the results
                - `scp -r test_nbx $om:$omx`
                - `ssh $om sbatch -D $omx/test_nbx $omx/test_nbx/run.sh`
                - `scp -r $om:$omx/test_nbx/results ./results`
            
        
        
        </div>
        
        </div>
        <div class="codecell" markdown="1">
        <div class="input_area" markdown="1">
        
        ```python
        !python test_nbx/experiment.py
        ```
        
        </div>
        <div class="output_area" markdown="1">
        
            **nbx**
            Running Experiment...
            	kwargs: {}
            	y: 0
            	x: 0
            	results_dir: ./
            	task_id: 0
            my results: 0 0 0
            
            **nbx**
            Experiment finished.
        
        
        </div>
        
        </div>
        <div class="codecell" markdown="1">
        <div class="input_area" markdown="1">
        
        ```python
        !scp -r test_nbx $om:$omx
        ```
        
        </div>
        <div class="output_area" markdown="1">
        
            wrapper.py                                    100%  950   106.0KB/s   00:00    
            __init__.py                                   100%    0     0.0KB/s   00:00    
            run.sh                                        100%  732   117.9KB/s   00:00    
            experiment.py                                 100%  739    78.9KB/s   00:00    
        
        
        </div>
        
        </div>
        <div class="codecell" markdown="1">
        <div class="input_area" markdown="1">
        
        ```python
        !ssh $om sbatch -D $omx/test_nbx $omx/test_nbx/run.sh
        ```
        
        </div>
        <div class="output_area" markdown="1">
        
            Submitted batch job 15969897
        
        
        </div>
        
        </div>
        <div class="codecell" markdown="1">
        <div class="input_area" markdown="1">
        
        ```python
        !ssh $om squeue -u $omid
        ```
        
        </div>
        <div class="output_area" markdown="1">
        
                         JOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)
        
        
        </div>
        
        </div>
        
Keywords: openmind; mit
Platform: UNKNOWN
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Requires-Python: >=3.5
Description-Content-Type: text/markdown
