Metadata-Version: 2.1
Name: orbweaver-tools
Version: 1.3.5
Summary: Stealth-oriented utilities for web scraping with rotating user agents, proxies, and Selenium helpers.
Home-page: https://github.com/Tom3man/orb-weaver
License: MIT
Keywords: web-scraping,selenium,proxies,automation
Author: Tom Freeman
Requires-Python: >=3.10,<4.0
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Topic :: Software Development :: Libraries
Provides-Extra: all
Provides-Extra: env
Provides-Extra: scraping
Provides-Extra: selenium
Requires-Dist: bs4 (>=0.0.1,<0.0.2) ; extra == "scraping" or extra == "all"
Requires-Dist: fake-useragent (>=1.4.0,<2.0.0)
Requires-Dist: pandas (>=2.1.4,<3.0.0) ; extra == "scraping" or extra == "all"
Requires-Dist: python-dotenv (>=1.0.1,<2.0.0) ; extra == "env" or extra == "all"
Requires-Dist: requests (>=2.31.0,<3.0.0)
Requires-Dist: selenium (>=4.16.0,<5.0.0) ; extra == "selenium" or extra == "all"
Requires-Dist: webdriver-manager (>=4.0.1,<5.0.0) ; extra == "selenium" or extra == "all"
Project-URL: Documentation, https://tom3man.github.io/orb-weaver/
Project-URL: Issues, https://github.com/Tom3man/orb-weaver/issues
Project-URL: Repository, https://github.com/Tom3man/orb-weaver
Description-Content-Type: text/markdown

# SQLite Forge

SQLite Forge is a lightweight toolkit that helps you declare and maintain SQLite tables from Python. Define your schema once, then manage tables, run queries, ingest pandas `DataFrame` objects, and export results.

## Highlights

- Declarative table definitions with schemas and optional multi-column primary keys
- Safe helpers to create/drop tables and check existence
- DataFrame ingestion with optional incremental overwrite support
- Query execution that returns pandas `DataFrame` objects
- Table export helpers for `csv`, `json`, and `parquet`

## Installation

```bash
pip install sqlite-forge
```

For development:

```bash
git clone https://github.com/Tom3man/sqlite-forge.git
cd sqlite-forge
poetry install --with dev --with docs
```

## Quick Start

```python
from pathlib import Path

import pandas as pd

from sqlite_forge import SqliteDatabase


class ExampleTable(SqliteDatabase):
    DEFAULT_PATH = "example_table"
    PRIMARY_KEY = ("id",)
    DEFAULT_SCHEMA = {
        "id": "INTEGER",
        "name": "TEXT",
        "score": "REAL",
    }


db = ExampleTable(database_path=Path("./data"))
db.create_table(overwrite=True)

db.ingest_dataframe(
    pd.DataFrame(
        [
            {"id": 1, "name": "Alice", "score": 9.2},
            {"id": 2, "name": "Bob", "score": 8.7},
        ]
    )
)

print(db.fetch_table())
db.export_table("./data/example_table.csv", format="csv")
```

## Development

```bash
poetry run pytest
poetry run ruff check .
poetry run mypy
poetry build
```

## Documentation

- Docs site: https://tom3man.github.io/sqlite-forge/
- Build locally:

```bash
poetry run mkdocs serve
```

## Release

1. Bump version in `pyproject.toml`.
2. Update `CHANGELOG.md`.
3. Publish:

```bash
poetry publish --build
```

## Changelog

See [CHANGELOG.md](CHANGELOG.md).

## Licence

MIT. See [LICENSE](LICENSE).

