|build-status|

SageCreator
===========

| SageCreator is a package meant to simplify cluster setup for Machine Learning in AWS.
| It does all the heavy lifting to get cluster up and running in a matter of minutes on any of the AWS instance type(s).
| It is using **spot instances** by default which can significantly reduce total cost of running the cluster.
| If spot instances are not available or the specified spot instance price is too low, it falls back to **on-demand** instances.
|
| After provision you can access **Jupyter notebook** that can run your code against provisioned server(s)

Installation
------------

Install and update using `pip`_:

.. code-block:: text

    $ pip install sagecreator

Python is required and it is highly recommended to install and run the package in `virtualenv`_.
Supported in Python >= 3.5.0.

.. _pip: https://pip.pypa.io/en/stable/quickstart/

.. _virtualenv: https://virtualenv.pypa.io/en/stable/


Prerequisites
-------------

**AWS Account**

To provision the cluster you need an `AWS Account`_  and `IAM user`_ either in Administrators group or with custom policy:

- Access Key ID
- Secret Access Key
- Key pair (optional) - if not provided it will be generated and stored locally

Execution
---------

After the installation, configure the tool by specifying configuration parameters:

.. code-block:: text

    $ sage configure
    Access key id: <AWS Access Key ID>
    Secret access key: <AWS Secret Access Key>
    Company: <Name of your organization>
    Owner: <Name of your team>
    Private key file: <Absolute path to private key file> (optional, if not provided will be generated by the key pair)

| **Company**, **Owner**, **Service** are required - those are used as tags for each instance in the cluster.
| **Private key file** - if provided, given private key file will be used to provision the cluster.

---------

| Provision the cluster.
| Provision step can take up to 20 minutes depending on network connection, cluster size, and instance type.
|

.. code-block:: text

    $ sage provision
    Service: <Name of your service>
    Instance type [t3.small]: <Instance type> (optional, if not provided default t3.small will be used)
    Spot instance price [0.1]: <Spot instance price> (optional, if not provided default price of $0.1 will be used)
    Cluster size [1]: <Cluster size> (optional, if not provided default cluster with 1 node will be provisioned)

.. image:: https://s3.amazonaws.com/evoneutron/github/sagecreator/provision1080.gif

| **Important**:
| The tool provides no guarantee that the instance(s) will be provisioned at specified **Spot instance price**.
| If specified Spot instance price is lower than current AWS spot instance price then **on-demand** instance(s) will be provisioned.
| Thus, it is up to the user to ensure that specified price is high enough for the request to be fulfilled.
| The tool does provide guarantee that the cluster with specified configuration will be provisioned and thus it has to fall back to **on-demand** instances if it can't provision spot instances.

---------

| Show path of the cluster configuration file.
| Though not necessary it is possible to manually edit that file with customizations prior to running **provision** step.
|

.. code-block:: text

    $ sage pwd

---------

Terminate cluster. This operation terminates all cluster nodes matching tags tuple of **Company**, **Owner**, **Service**.

.. code-block:: text

    $ sage terminate

Jupyter access
--------------

| Once provision step is done and cluster is up you can access jupyter notebook on http://localhost:9000
| We have provided a sample notebook to train a model on Fashion MNIST dataset using CNN in Keras

Under the hood
--------------

| The logic that orchestrates the cluster and deploys the software is written in **Ansible**
|


.. |build-status| image:: https://travis-ci.com/evoneutron/sagecreator.svg?branch=master
    :target: https://travis-ci.com/evoneutron/sagecreator

.. _`AWS Account`: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/get-set-up-for-amazon-ec2.html#sign-up-for-aws

.. _`IAM User`: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/get-set-up-for-amazon-ec2.html#create-an-iam-user
