# -*- coding: utf-8 -*-
from setuptools import setup

packages = \
['dbt_airflow', 'dbt_airflow.core', 'dbt_airflow.domain', 'dbt_airflow.parser']

package_data = \
{'': ['*']}

install_requires = \
['apache-airflow>=2.0.0,<3', 'dbt-core>=1.2.0', 'pydantic>=1.10.4,<2.0.0']

setup_kwargs = {
    'name': 'dbt-airflow',
    'version': '0.1.0',
    'description': 'A Python package that creates fine-grained Airflow tasks for dbt',
    'long_description': '# dbt-airflow\nA Python package that helps Data and Analytics engineers render dbt projects on Apache Airflow\nin a way that every single dbt resource (model, test, seed and snapshot) is represented by a single\nAirflow task. \n\n### Features\n- Render dbt project as a TaskGroup in Airflow Tasks\n- Every `model`, `seed` and `snapshot` resource that has at least a single test, will also have a corresponding\ntest task as a downstream.\n- Add extra tasks before or after the whole dbt project\n- Add extra tasks after specific dbt tasks\n\n## How does it work\nThe library essentially builds on top of the metadata generated by `dbt-core` and are stored in \nthe `target/manifest.json` file in your dbt project directory.  \n\n## Domain Requirements\n\n- Every dbt project, when compiled, will generate a metadata file under `<dbt-project-dir>/target/manifest.json`\n- The manifest file contains information about the interdependencies of the project\'s data models\n- `dbt-airflow` aims to extract these dependencies such that every dbt entity (snapshot, model, test and seed) has \n  its own task in a Airflow DAG while entity dependencies are persisted\n- Snapshots are never an upstream dependency of any task\n- The creation of snpashots on seeds does not make sense, and thus not handled \n  not even sure if this is even possible on dbt side)\n- Models may have tests\n- Snapshots may have tests\n- Seeds may have tests\n\n---\n\n# Installation\n\nThe package is available on PyPI and can be installed through `pip`:\n```bash\npip install dbt-airflow\n```\n\n`dbt` needs to connect to your target environment (database, warehouse etc.) and in order to do so, it makes use of \ndifferent adapters, each dedicated to a different technology (such as Postgres or BigQuery). Therefore, before running\n`dbt-airflow` you also need to ensure that the required adapter(s) are installed in your environment. \n\nFor the full list of available adapters please refer to the official \n[dbt documentation](https://docs.getdbt.com/docs/available-adapters). \n\n---\n\n# Usage\n`dbt-airflow` can be used either as a normal Python package, or through the \ncommand line interface. \n\nGiven that there are possibly many different ways for deploying Airflow and automating different aspects\nof data workflows that involve Airflow, dbt and potentially other tools as well, we wanted to offer more\nflexibility by providing different approaches for using `dbt-airflow`.\n\n### Building an Airflow DAG using `dbt-airflow`\n\n```python3\nfrom datetime import datetime\nfrom pathlib import Path\n\nfrom airflow import DAG\nfrom airflow.operators.python import PythonOperator\nfrom airflow.operators.dummy import DummyOperator\n\nfrom dbt_airflow.core.task_group import DbtTaskGroup\nfrom dbt_airflow.core.task import ExtraTask\n\n\nwith DAG(\n    dag_id=\'test_dag\',\n    start_date=datetime(2021, 1, 1),\n    catchup=False,\n    tags=[\'example\'],\n) as dag:\n\n    extra_tasks = [\n        ExtraTask(\n            task_id=\'test_task\',\n            operator=PythonOperator,\n            operator_args={\n                \'python_callable\': lambda: print(\'Hello world\'),\n            },\n            upstream_task_ids={\n                \'model.example_dbt_project.int_customers_per_store\',\n                \'model.example_dbt_project.int_revenue_by_date\'\n            }\n        ),\n        ExtraTask(\n            task_id=\'another_test_task\',\n            operator=PythonOperator,\n            operator_args={\n                \'python_callable\': lambda: print(\'Hello world 2!\'),\n            },\n            upstream_task_ids={\n                \'test.example_dbt_project.int_customers_per_store\',\n            },\n            downstream_task_ids={\n                \'snapshot.example_dbt_project.int_customers_per_store_snapshot\',\n            }\n        ),\n        ExtraTask(\n            task_id=\'test_task_3\',\n            operator=PythonOperator,\n            operator_args={\n                \'python_callable\': lambda: print(\'Hello world 3!\'),\n            },\n            downstream_task_ids={\n                \'snapshot.example_dbt_project.int_customers_per_store_snapshot\',\n            },\n            upstream_task_ids={\n                \'model.example_dbt_project.int_revenue_by_date\',\n            },\n        )\n    ]\n\n    t1 = DummyOperator(task_id=\'dummy_1\')\n    t2 = DummyOperator(task_id=\'dummy_2\')\n\n    tg = DbtTaskGroup(\n        group_id=\'dbt-company\',\n        dbt_manifest_path=Path(\'/opt/airflow/example_dbt_project/target/manifest.json\'),\n        dbt_target=\'dev\',\n        dbt_project_path=Path(\'/opt/airflow/example_dbt_project/\'),\n        dbt_profile_path=Path(\'/opt/airflow/example_dbt_project/profiles\'),\n        extra_tasks=extra_tasks,\n        create_sub_task_groups=True,\n    )\n\n    t1 >> tg >> t2\n\n```\n\n# Contributing\nIf you would like to contribute to `dbt-airflow` project, you will essentially need to follow the steps outlined below:\n1. Create a fork of the repository\n2. Set up the development environment on your local machine (see the detailed guide below)\n3. Write and test your contribution\n4. Create a Pull Request\n\n##  Setting up your local development environment\n\nTO BE FINALISED\n```bash\ndocker-compose build\n\ndocker-compose up\n\n# Access postgres db (changed to port 5433 given that we have an additional postgres instance for Airflow itsefl)\ndocker ps  # get the container id of postgres-sakila\ndocker exec -it <container-id> /bin/bash\npsql -U postgres -p 5433 \n```\n\n```bash\n# Install poetry\npip install poetry\n\n# Install dependencies in poetry venv\npoetry install \n\n# Run tests\npoetry run tests -rP -vv\n\n# Run specific test(s)\npoetry run tests -k "test_name_or_prefix" -rP -vv\n```\n',
    'author': 'Giorgos Myrianthous',
    'author_email': 'giorgos.myrianthous@gmail.com',
    'maintainer': 'None',
    'maintainer_email': 'None',
    'url': 'https://github.com/gmyrianthous/dbt-airflow',
    'packages': packages,
    'package_data': package_data,
    'install_requires': install_requires,
    'python_requires': '>=3.7.2,<4',
}


setup(**setup_kwargs)
