Metadata-Version: 2.0
Name: scrapy-dotpersistence
Version: 0.2.2
Summary: Scrapy extension to sync `.scrapy` folder to an S3 bucket
Home-page: https://github.com/scrapy-plugins/scrapy-dotpersistence
Author: Scrapy developers
Author-email: opensource@scrapinghub.com
License: BSD
Platform: Any
Classifier: Development Status :: 5 - Production/Stable
Classifier: License :: OSI Approved :: BSD License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 2
Classifier: Programming Language :: Python :: 2.7
Requires-Dist: Scrapy (>=1.0.3)
Requires-Dist: s3cmd (>=1.6.0)

=====================
scrapy-dotpersistence
=====================

Scrapy extension to sync `.scrapy` folder to an S3 bucket.

Installation
============

You can install scrapy-dotpersistence using pip::

    pip install scrapy-dotpersistence

You can then enable the extension in your `settings.py`::

    EXTENSIONS = {
        ...
        'scrapy_dotpersistence.DotScrapyPersistence': 0
    }

How to use it
=============

Enable extension through `settings.py`::

    DOTSCRAPY_ENABLED = True

Configure the exension through `settings.py`::

    ADDONS_AWS_ACCESS_KEY_ID = "ABC"
    ADDONS_AWS_SECRET_ACCESS_KEY = "DEF"
    ADDONS_AWS_USERNAME = "username"
    ADDONS_S3_BUCKET = "test-bucket-name"

You can change a dotpersistence folder path with environ::

    export DOTSCRAPY_DIR='/tmp/.scrapy'


