Metadata-Version: 2.1
Name: kedroio
Version: 0.1.2
Summary: Extension for `kedro` datasets
Home-page: https://github.com/atsangarides/kedroio
License: MIT
Keywords: data pipelines,kedro
Author: atsangarides
Author-email: andreas_tsangarides@hotmail.com
Requires-Python: >=3.8,<4.0
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Dist: boto3 (>=1.20.17,<2.0.0)
Requires-Dist: boto3-stubs[s3] (>=1.20.24,<2.0.0)
Requires-Dist: tqdm (>=4.62.3,<5.0.0)
Project-URL: Repository, https://github.com/atsangarides/kedroio
Description-Content-Type: text/markdown

# kedroio
A module extending the datasets that come shipped with `kedro`

[![](https://img.shields.io/badge/python-3.8-blue.svg)](https://github.com/pyenv/pyenv)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white)](https://github.com/pre-commit/pre-commit)

## Example usage
```sql
-- example.sql
select *
from "database"."table_name"
limit 5;
```

```yaml
# conf/base/catalog.py
my_athena_dataset:
  type: kedroio.datasets.aws.athena.AthenaQueryDataSet
  filepath: data/01_raw/example.csv
  sql_filepath: example.sql
  bucket: example-bucket
  workgroup: primary
  subfolder: data
  region_name: eu-west-2
  read_result: true # read into pandas DataFrame
  overwrite: false # skip download if filepath exists
```

## Testing

Start `moto` server for mocked AWS resources
```sql
moto_server
```

Run tests
```sql
pytest tests/
```
