# crawl4ai-client

Lightweight async Python client for [Crawl4AI](https://github.com/unclecode/crawl4ai) Docker server.

**No browser dependencies required.** Just `httpx` + `pydantic` (~2MB vs ~500MB for the full crawl4ai package).

## Install

```bash
pip install crawl4ai-client
```

## Quick Start

```python
import asyncio
from crawl4ai_client import Crawl4aiDockerClient

async def main():
    async with Crawl4aiDockerClient(
        base_url="http://localhost:11235",
        api_token="your-token",  # optional
    ) as client:
        result = await client.crawl(["https://example.com"])
        print(result.raw_markdown)

asyncio.run(main())
```

## Features

- **Crawl** single or multiple URLs
- **Stream** results as they complete
- **Get markdown** via the `/md` endpoint
- **Screenshots** and schema retrieval
- **Per-URL configs** for batch crawling (crawler_configs list)
- **Async context manager** with automatic cleanup

## Usage

### Basic crawl

```python
from crawl4ai_client import Crawl4aiDockerClient, CrawlerRunConfig, CacheMode

async with Crawl4aiDockerClient(base_url="http://localhost:11235") as client:
    result = await client.crawl(
        ["https://example.com"],
        crawler_config=CrawlerRunConfig(cache_mode=CacheMode.BYPASS),
    )
    print(result.raw_markdown)
```

### Multiple URLs with per-URL configs

```python
from crawl4ai_client import Crawl4aiDockerClient, CrawlerRunConfig

async with Crawl4aiDockerClient(base_url="http://localhost:11235") as client:
    results = await client.crawl(
        ["https://example.com", "https://httpbin.org/html"],
        crawler_configs=[
            CrawlerRunConfig(word_count_threshold=5),
            CrawlerRunConfig(word_count_threshold=50),
        ],
    )
    for r in results:
        print(f"{r.url}: {len(r.raw_markdown)} chars")
```

### Streaming

```python
async with Crawl4aiDockerClient(base_url="http://localhost:11235") as client:
    async for result in client.crawl_stream(["https://example.com", "https://httpbin.org/html"]):
        print(f"Got: {result.url}")
```

### Markdown endpoint

```python
async with Crawl4aiDockerClient(base_url="http://localhost:11235") as client:
    md = await client.get_markdown("https://example.com", content_filter="fit")
    print(md)
```

## Why this package?

The full `crawl4ai` package installs 34+ dependencies (~500MB) including Playwright, browsers, numpy, and litellm. If you're running Crawl4AI as a Docker service and only need the client, this package gives you the same `Crawl4aiDockerClient` with just 2 dependencies.

## Compatibility

This client is compatible with Crawl4AI Docker server v0.8.x+. The config classes (`BrowserConfig`, `CrawlerRunConfig`) produce the same serialized format as the full library.

## License

Apache 2.0 — based on [crawl4ai](https://github.com/unclecode/crawl4ai) by [unclecode](https://github.com/unclecode).
