Metadata-Version: 2.1
Name: datamechanics-airflow-plugin
Version: 1.1.1
Summary: An Airflow plugin to launch and monitor Spark applications on the Data Mechanics platform
Home-page: https://github.com/datamechanics/datamechanics_airflow_plugin
Author: Data Mechanics Engineering Team
Author-email: engineering@datamechanics.co
License: Apache Software License 2.0
Keywords: datamechanics_airflow_plugin
Platform: UNKNOWN
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Requires-Python: >=3.5
Description-Content-Type: text/markdown
Requires-Dist: requests
Requires-Dist: apache-airflow (>=1.10.0)

# Data Mechanics Airflow Plugin

An Airflow plugin to launch and monitor Spark applications on the Data Mechanics platform.

## Environment

* Python >= 3.5
* apache-airflow >= 1.10.x. Compatible with Airflow 2.

## Installation and usage

A tutorial to configure and use this plugin is available in the [Data Mechanics docs](https://docs.datamechanics.co/docs/airflow-plugin).

The main difference between Airflow 1 and Airflow 2 is how to import the plugin:
```python
# Airflow 1
from airflow.operators.datamechanics import DataMechanicsOperator

# Airflow 2
from datamechanics_airflow_plugin.operator import DataMechanicsOperator
```

## Example DAGs

You can see example DAGs for [Airflow 1](https://github.com/datamechanics/datamechanics_airflow_plugin/tree/master/example_dags_airflow1) and [Airflow 2](https://github.com/datamechanics/datamechanics_airflow_plugin/tree/master/example_dags_airflow2).

## Development

[Development instructions](DEVELOPMENT.md).


# Changelog

## [1.1.1] 2021-03-26

### Fixed

- Kill an app instead of deleting it when the Airflow task is marked as failed

## [1.1.0] 2021-03-10

### Added

- Support for Airflow templating in Data Mechanics operator's arguments (by @jj-ookla)
- Minimal support for Airflow 2 (the code is unchanged, only the doc and the contributor dev environment have changed)

## [1.0.7] 2020-09-11

### Changed

- Changed bumversion config

## [1.0.6] 2020-09-11

### Changed

- Updated doc

## [1.0.5] 2020-09-11

### Fixed

- Dependency management

## [1.0.0] 2020-09-11

### Changed

- Converted the existing plugin into a Python package

[unreleased]: https://github.com/datamechanics/datamechanics_airflow_plugin/compare/v1.1.1...master
[1.1.1]: https://github.com/datamechanics/datamechanics_airflow_plugin/compare/v1.1.0...v1.1.1
[1.1.0]: https://github.com/datamechanics/datamechanics_airflow_plugin/compare/v1.0.7...v1.1.0
[1.0.7]: https://github.com/datamechanics/datamechanics_airflow_plugin/compare/v1.0.6...v1.0.7
[1.0.6]: https://github.com/datamechanics/datamechanics_airflow_plugin/compare/v1.0.5...v1.0.6
[1.0.5]: https://github.com/datamechanics/datamechanics_airflow_plugin/compare/v1.0.0...v1.0.5
[1.0.0]: https://github.com/datamechanics/datamechanics_airflow_plugin/compare/...v1.0.0


