Metadata-Version: 2.1
Name: scrapyq
Version: 1.0.0
Summary: A library to filter SQLAlchemy queries.
Home-page: https://github.com/slymit/scrapyq
Author: slymit
Author-email: slymit@gmail.com
Maintainer: slymit
Maintainer-email: slymit@gmail.com
License: BSD
Classifier: License :: OSI Approved :: BSD License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Console
Classifier: Environment :: No Input/Output (Daemon)
Classifier: Topic :: Internet :: WWW/HTTP
Description-Content-Type: text/x-rst
Requires-Dist: scrapyd==1.4.2
Requires-Dist: redis==5.0.0
Provides-Extra: dev
Requires-Dist: pytest==7.4.2; extra == "dev"
Requires-Dist: packaging==23.1; extra == "dev"
Requires-Dist: coverage>=7.2.7; extra == "dev"
Requires-Dist: flake8; extra == "dev"
Requires-Dist: restructuredtext-lint; extra == "dev"

scrapyq
=======

Scrapyq is designed to replace the SQLite backend by a Redis backend.
In other words, all the queue management will be done using Redis.

Scrapyq is a fork of the original
https://github.com/speakol-ads/scrapyd-redis implementation.


Install
-------

.. code-block::

    pip install scrapyq


Config
------

To start using this library you just need to override
the ``spiderqueue`` option in your ``scrapyd.conf`` file:

.. code-block::

    [scrapyd]
    spiderqueue = scrapyq.spiderqueue.RedisSpiderQueue
    ...

If you want to customize the access to the database,
you can add into your ``scrapyd.conf`` file:

.. code-block::

    [scrapyq]
    queue_prefix = scrapyq.queue.
    redis_db = 0
    redis_host = localhost
    redis_port = 6379
    redis_username = 'admin'  # (Optional)
    redis_password = 'password'  # (Optional)
    ...
