Metadata-Version: 1.1
Name: aiscalator
Version: 0.1.1
Summary: AIscalate your Jupyter Notebook Prototypes into Airflow Data Products
Home-page: https://github.com/Aiscalate/aiscalator
Author: Christophe Duong
Author-email: chris@aiscalate.com
License: Apache Software License 2.0
Description: ==========
        AIscalator
        ==========
        
        
        
        
        
        * Free software: Apache Software License 2.0
        * Website: http://www.aiscalate.com
        * Documentation: https://aiscalator.readthedocs.io.
        * Bugs: https://github.com/aiscalate/aiscalator/issues
        
        Key Features
        ------------
        
        Aiscalator is a toolbox to enable your team streamlining
        processes from innovation to productization with:
        
        * Jupyter workbench
            * Explore Data, Prototype Solutions
        * Docker wrapper tools
            * Share Code, Deploy Reproducible Environments
        * Airflow machinery
            * Schedule Tasks, Refine Products
        * Data Science and Data Engineering best practices
        
        .. image:: _static/aiscalator_process.png
                :target: _static/aiscalator_process.png
                :align: center
                :alt: From Prototype to Production Workflow
        
        ===========
        Quick Start
        ===========
        
        Installation
        ------------
        
        Test if prerequisite softwares are installed:
        
        .. code-block:: shell
        
            docker --version
            docker-compose --version
            pip --version
        
        Install AIscalator tool::
        
            git clone https://github.com/Aiscalate/aiscalator.git
            cd aiscalator/
            make install
        
        Great, we are now ready to use the AIscalator!
        
        The following setup commands are completely optional because they are dealing with
        prebuilding Docker images. If you choose not to do it at this point, they
        will get built later on whenever they are required.
        
        However, since producing a Docker image requires a certain amount of time
        to download, install packages, and sometimes even compiling them, these
        installation steps can be initiated right away all at once. Thus, you
        should be free to go enjoy a nice coffee break!
        
        You might want to customize your environment with the AIscalator, this
        will ask you few questions::
        
            aiscalator setup
        
        Build docker images to run Jupyter environments::
        
            aiscalator jupyter setup
        
        Build docker image to run Airflow::
        
            # aiscalator airflow setup <path-to-workspace-folder>
            # for example,
            aiscalator airflow setup $PWD
        
        
        Start working
        -------------
        
        AIscalator commands dealing with jupyter are defining tasks in Airflow jargon;
        In our case, they are all wrapped inside a Docker container. We also refer to
        them as Steps.
        
        Whereas AIscalator commands about airflow are made to author, schedule and monitor
        DAGs (Directed Acyclic Graphs). They define how a workflow is composed of multiple
        steps, their dependencies and execution times or triggers.
        
        Jupyter
        -------
        
        Create a new Jupyter notebook to work on, define corresponding AIscalator step::
        
            # aiscalator jupyter new <path-to-store-new-files>
            # For example,
            aiscalator jupyter new project
            # (CTRL + c to kill when done)
        
        Or you can edit an existing AIscalator step::
        
            # aiscalator jupyter edit <aiscalator step>
            # For example, if you cloned the git repository:
            aiscalator jupyter edit resources/example/example.conf
            # (CTRL + c to kill when done)
        
        Run the step without GUI::
        
            # aiscalator jupyter run <aiscalator task>
            # For example, if you cloned the git repository:
            aiscalator jupyter run resources/example/example.conf
        
        Airflow
        -------
        
        Start Airflow services::
        
            aiscalator airflow start
        
        Create a new AIscalator DAG, define the airflow job::
        
            # aiscalator airflow new <path-to-store-new-files>
            # For example,
            aiscalator airflow new project
            # (CTRL + c to kill when done)
        
        Or you can edit an existing AIscalator DAG::
        
            # aiscalator airflow edit <aiscalator DAG>
            # For example, if you cloned the git repository:
            aiscalator airflow edit resources/example/example.conf
            # (CTRL + c to kill when done)
        
        Schedule AIscalator DAG into local airflow dags folder::
        
            # aiscalator airflow push <aiscalator DAG>
            # For example, if you cloned the git repository:
            aiscalator airflow push resources/example/example.conf
        
        Stop Airflow services::
        
            aiscalator airflow stop
        
        =======
        History
        =======
        
        0.1.0 (2018-11-07)
        ------------------
        
        * First Alpha release on PyPI.
        
Keywords: data science,jupyter,jupyter notebook,prototype,papermill,jupytext,data engineering,product,airflow
Platform: UNKNOWN
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: System Administrators
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Natural Language :: English
Classifier: Operating System :: Unix
Classifier: Operating System :: POSIX
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.4
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: Implementation :: PyPy
Classifier: Topic :: Utilities
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Application Frameworks
