Metadata-Version: 2.4
Name: scrapyduler
Version: 2.0.0
Summary: Scrapyd launcher module that schedules scrapy spiders by time
Author-email: Dmitriy Chuenkov <slymit@gmail.com>
License: BSD-3-Clause
Project-URL: Homepage, https://github.com/slymit/scrapyduler
Keywords: scrapy,scrapyd,scheduler,launcher
Classifier: License :: OSI Approved :: BSD License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Environment :: Console
Classifier: Environment :: No Input/Output (Daemon)
Classifier: Topic :: Internet :: WWW/HTTP
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: scrapyd~=1.6.0
Requires-Dist: APScheduler==3.11.2
Requires-Dist: importlib-metadata; python_version >= "3.7"
Provides-Extra: dev
Requires-Dist: pytest>=9.0.2; extra == "dev"
Requires-Dist: pytest-mock>=3.15.1; extra == "dev"
Requires-Dist: pytest-cov>=7.0.0; extra == "dev"
Requires-Dist: ruff>=0.15.0; extra == "dev"
Requires-Dist: mdformat>=1.0.0; extra == "dev"

# scrapyduler

[![version](https://img.shields.io/pypi/v/scrapyduler.svg)](https://pypi.python.org/pypi/scrapyduler)
[![pyversions](https://img.shields.io/pypi/pyversions/scrapyduler.svg)](https://pypi.python.org/pypi/scrapyduler)
[![actions](https://github.com/slymit/scrapyduler/actions/workflows/python-test.yml/badge.svg)](https://github.com/slymit/scrapyduler/actions/workflows/python-test.yml)
[![codecov](https://codecov.io/github/slymit/scrapyduler/graph/badge.svg?token=H1SMMJ0JZ7)](https://codecov.io/github/slymit/scrapyduler)

Scrapyd launcher module that schedules scrapy spiders by time.

## Install

```shell
pip install scrapyduler
```

## Config

To start using this library you just need to override
the `launcher` option in your `scrapyd.conf` file:

```ini
[scrapyd]
launcher = scrapyduler.launcher.Launcher
```

and then add the schedulers configuration, e.g.:

```ini
[scheduler.1]
cron        = * * * * *
project     = quotesbot
spider      = toscrape-xpath
settings    = HTTPPROXY_ENABLED=True
args        = key1=value1 key2=value2 start_url=http://quotes.toscrape.com/

[scheduler.2]
interval    = weeks=0 days=0 hours=0 minutes=0 seconds=30
project     = quotesbot
spider      = toscrape-css
settings    = HTTPPROXY_ENABLED=True
args        = key1=value1 key2=value2 start_url=http://quotes.toscrape.com/
```

In the examples above, we set up two schedulers.
The first scheduler uses cron syntax to run spiders.
The second scheduler triggers on specified intervals,
starting on `start_date` if specified, `datetime.now()` + interval otherwise.
See <https://github.com/agronholm/apscheduler> for more information.
