Metadata-Version: 1.1
Name: sparkonda
Version: 0.2.2
Summary: Minimalistic utility library to manage conda environments for pyspark jobs on yarn clusters
Home-page: https://github.com/moutai/sparkonda
Author: Moussa Taifi
Author-email: @moutai
License: ISCL
Description: ===============================
        Sparkonda
        ===============================
        
        Minimalistic utility library to manage conda environments for PySpark jobs on yarn clusters.
        
        Features
        --------
        
        * Manage conda environments on PySpark executors to use specific
          packages on the remote workers without involving admins to install needed software on hadoop cluster.
        
        Docs
        --------
        
        http://sparkonda.readthedocs.org
        
        
        
        
        History
        -------
        
        0.2.2 (2016-02)
        ---------------------
        Fix bug when number of cores per executor is larger than 1
        
        Fix list_cwd to include hostname in output
        
        Fix tests to use pack files naming instead of zip files
        
        Fix usage docs to include new config for number of cores
        
        Add tests for correct prun to num partition mapping
        
        0.2.1 (2016-01)
        ---------------------
        Fix documentation, setup and usage methods
        
        Fix travis and setup.py configs
        
        Move zip to pack as an action
        
        Replace zip with tar to better preserve acls on conda env files
        
        Add configuration of error level for untar
        
        0.2.0 (2016-01)
        ---------------------
        Additional tests for the distributed version of remote package delivery
        
        Changed os files management to python based support (zip, rm)
        
        Use SparkFiles to detect files distributed to the workers.
        
        0.1.0 (2015-11)
        ---------------------
        Initial version to manage the conda environments on pyspark cluster workers without involving your cluster admins too heavily.
        
        
        
Keywords: sparkonda
Platform: UNKNOWN
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: ISC License (ISCL)
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 2.7
