Metadata-Version: 2.4
Name: aioscraper
Version: 0.8.0
Summary: Async framework for building modular and scalable web scrapers.
Author: darkstussy
Project-URL: Homepage, https://github.com/darkstussy/aioscraper
Project-URL: Issues, https://github.com/darkstussy/aioscraper/issues
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Framework :: AsyncIO
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Topic :: Internet :: WWW/HTTP :: Indexing/Search
Classifier: Topic :: Software Development :: Libraries :: Application Frameworks
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: aiohttp[speedups]~=3.13.2
Requires-Dist: aiojobs~=1.4.0
Provides-Extra: uvloop
Requires-Dist: uvloop~=0.22.1; extra == "uvloop"
Provides-Extra: dev
Requires-Dist: flake8~=7.3.0; extra == "dev"
Requires-Dist: black~=25.11.0; extra == "dev"
Requires-Dist: pyright~=1.1.407; extra == "dev"
Requires-Dist: aiohttp[speedups]~=3.13.2; extra == "dev"
Requires-Dist: aiojobs~=1.4.0; extra == "dev"
Provides-Extra: test
Requires-Dist: pytest~=9.0.1; extra == "test"
Requires-Dist: pytest-asyncio~=1.3.0; extra == "test"
Requires-Dist: aresponses~=3.0.0; extra == "test"
Dynamic: license-file

# aioscraper

<p align="center">
  <img src="docs/static/aioscraper.png" alt="aioscraper logo" width="340">
</p>

### Asynchronous framework for building modular and scalable web scrapers.

![Python](https://img.shields.io/badge/python-3.11%2B-blue)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
![PyPI](https://img.shields.io/pypi/v/aioscraper)
![Tests](https://github.com/darkstussy/aioscraper/actions/workflows/tests.yml/badge.svg)
[![Documentation Status](https://readthedocs.org/projects/aioscraper/badge/?version=latest)](https://aioscraper.readthedocs.io/en/latest/)
![GitHub last commit](https://img.shields.io/github/last-commit/darkstussy/aioscraper)

> **Beta notice:** APIs and behavior may change; expect sharp edges while things settle.

## Key Features

- Fully asynchronous architecture powered by `aiohttp` and `aiojobs`
- Modular system with middleware support
- Pipeline data processing
- Flexible configuration
- Priority-based request queue management
- Built-in error handling

[Documentation](https://aioscraper.readthedocs.io)

## Getting started

Install
```bash
pip install aioscraper
```

Create `scraper.py`:
```python
from aioscraper import AIOScraper
from aioscraper.types import Request, Response, SendRequest

scraper = AIOScraper()

@scraper.register
async def scrape(send_request: SendRequest):
    await send_request(Request(url="https://example.com", callback=handle_response))


async def handle_response(response: Response):
    print(f"Fetched {response.url} with status {response.status}")
```

Run it
```bash
aioscraper scraper
```

## License

MIT License

Copyright (c) 2025 darkstussy
