Metadata-Version: 2.1
Name: ingestr
Version: 0.1.2
Summary: ingestr is a command-line application that ingests data from various sources and stores them in any database.
Project-URL: Homepage, https://github.com/bruin-data/ingestr
Project-URL: Issues, https://github.com/bruin-data/ingestr/issues
Author-email: Burak Karakan <burak.karakan@getbruin.com>
License-File: LICENSE.md
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Database
Requires-Python: >=3.9
Requires-Dist: cx-oracle==8.3.0
Requires-Dist: databricks-sql-connector==2.9.3
Requires-Dist: dlt==0.4.3
Requires-Dist: duckdb-engine==0.11.1
Requires-Dist: duckdb==0.9.2
Requires-Dist: google-cloud-bigquery-storage==2.24.0
Requires-Dist: pendulum==3.0.0
Requires-Dist: psycopg2-binary==2.9.9
Requires-Dist: py-machineid==0.5.1
Requires-Dist: pymysql==1.1.0
Requires-Dist: pyodbc==5.1.0
Requires-Dist: redshift-connector==2.1.0
Requires-Dist: rich==13.7.0
Requires-Dist: rudder-sdk-python==2.0.2
Requires-Dist: snowflake-sqlalchemy==1.5.1
Requires-Dist: sqlalchemy-bigquery==1.9.0
Requires-Dist: sqlalchemy-redshift==0.8.14
Requires-Dist: sqlalchemy2-stubs==0.0.2a38
Requires-Dist: sqlalchemy==1.4.51
Requires-Dist: tqdm==4.66.2
Requires-Dist: typer==0.9.0
Description-Content-Type: text/markdown

<div align="center">
    <img src="https://github.com/bruin-data/ingestr/blob/main/resources/ingestr.svg?raw=true" width="500" />
    <p>Ingest & copy data from any source to any destination without any code</p>
    <img src="https://github.com/bruin-data/ingestr/blob/main/resources/demo.gif?raw=true" width="750" />
</div>

<div align="center" style="margin-top: 24px;">
  <a target="_blank" href="https://join.slack.com/t/bruindatacommunity/shared_invite/zt-2dl2i8foy-bVsuMUauHeN9M2laVm3ZVg" style="background:none">
    <img src="https://img.shields.io/badge/slack-join-dlt.svg?color=d95f5f&logo=slack" style="width: 180px;"  />
  </a>
</div>

-----

Ingestr is a command-line application that allows you to ingest data from any source into any destination using simple command-line flags, no code necessary.

- ✨ copy data from your database into any destination
- ➕ incremental loading: `append`, `merge` or `delete+insert`
- 🐍 single-command installation

ingestr takes away the complexity of managing any backend or writing any code for ingesting data, simply run the command and watch the data land on its destination.


## Installation
```
pip install ingestr
```

## Quickstart

```bash
ingestr ingest \
    --source-uri 'postgresql://admin:admin@localhost:8837/web?sslmode=disable' \
    --source-table 'public.some_data' \
    --dest-uri 'bigquery://<your-project-name>?credentials_path=/path/to/service/account.json' \
    --dest-table 'ingestr.some_data'
```

That's it.

This command will:
- get the table `public.some_data` from the Postgres instance.
- upload this data to your BigQuery warehouse under the schema `ingestr` and table `some_data`.

## Documentation
You can see the full documentation [here](https://bruin-data.github.io/ingestr/getting-started/quickstart.html).

## Community
Join our Slack community [here](https://join.slack.com/t/bruindatacommunity/shared_invite/zt-2dl2i8foy-bVsuMUauHeN9M2laVm3ZVg).


## Supported Sources & Destinations

| Database             | Source | Destination |
|----------------------|--------|-------------|
| Postgres             | ✅      | ✅         |
| BigQuery             | ✅      | ✅         |
| Snowflake            | ✅      | ✅         |
| Redshift             | ✅      | ✅         |
| Databricks           | ✅      | ✅         |
| DuckDB               | ✅      | ✅         |
| Microsoft SQL Server | ✅      | ✅         |
| Oracle               | ✅      | ❌         |
| SQLite               | ✅      | ❌         |
| MySQL                | ✅      | ❌         |
| Local CSV file       | ❌      | ✅         |

More to come soon!

## Acknowledgements
This project would not have been possible without the amazing work done by the [SQLAlchemy](https://www.sqlalchemy.org/) and [dlt](https://dlthub.com/) teams. We relied on their work to connect to various sources and destinations, and built `ingestr` as a simple, opinionated wrapper around their work.