Metadata-Version: 2.1
Name: flytekit
Version: 0.16.0b4
Summary: Flyte SDK for Python
Home-page: https://github.com/lyft/flytekit
Maintainer: Lyft
Maintainer-email: flyte-eng@lyft.com
License: apache2
Platform: UNKNOWN
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.7
Description-Content-Type: text/markdown
Requires-Dist: flyteidl (<1.0.0,>=0.18.11)
Requires-Dist: pandas (<2.0.0,>=1.0.0)
Requires-Dist: pyarrow (<3.0.0,>=2.0.0)
Requires-Dist: click (<8.0,>=6.6)
Requires-Dist: croniter (<4.0.0,>=0.3.20)
Requires-Dist: deprecated (<2.0,>=1.0)
Requires-Dist: python-dateutil (<=2.8.1,>=2.1)
Requires-Dist: grpcio (<2.0,>=1.3.0)
Requires-Dist: protobuf (<4,>=3.6.1)
Requires-Dist: pytimeparse (<2.0.0,>=1.1.8)
Requires-Dist: pytz (<2018.5,>=2017.2)
Requires-Dist: keyring (>=18.0.1)
Requires-Dist: requests (<3.0.0,>=2.18.4)
Requires-Dist: responses (>=0.10.7)
Requires-Dist: six (<2.0.0,>=1.9.0)
Requires-Dist: sortedcontainers (>=1.5.9<3.0.0)
Requires-Dist: statsd (<4.0.0,>=3.0.0)
Requires-Dist: urllib3 (<1.26,>=1.22)
Requires-Dist: wrapt (<2.0.0,>=1.0.0)
Requires-Dist: retry (==0.9.2)
Requires-Dist: dataclasses-json (>=0.5.2)
Requires-Dist: natsort (>=7.0.1)
Requires-Dist: dirhash (>=0.2.1)
Provides-Extra: all
Requires-Dist: pyspark (>=3.0.0) ; extra == 'all'
Requires-Dist: k8s-proto (<1.0.0,>=0.0.3) ; extra == 'all'
Requires-Dist: numpy (<2.0.0,>=1.14.0) ; extra == 'all'
Requires-Dist: pandas (<2.0.0,>=0.22.0) ; extra == 'all'
Requires-Dist: pyarrow (<3.0.0,>2.0.0) ; extra == 'all'
Requires-Dist: hmsclient (<1.0.0,>=0.0.1) ; extra == 'all'
Requires-Dist: papermill (>=1.2.0) ; extra == 'all'
Requires-Dist: nbconvert (>=6.0.7) ; extra == 'all'
Requires-Dist: ipykernel (>=5.0.0) ; extra == 'all'
Requires-Dist: black (==19.10b0) ; extra == 'all'
Requires-Dist: sagemaker-training (<4.0.0,>=3.6.2) ; extra == 'all'
Provides-Extra: all-spark2.4
Requires-Dist: pyspark (<3.0.0,>=2.4.0) ; extra == 'all-spark2.4'
Requires-Dist: k8s-proto (<1.0.0,>=0.0.3) ; extra == 'all-spark2.4'
Requires-Dist: numpy (<2.0.0,>=1.14.0) ; extra == 'all-spark2.4'
Requires-Dist: pandas (<2.0.0,>=0.22.0) ; extra == 'all-spark2.4'
Requires-Dist: pyarrow (<3.0.0,>2.0.0) ; extra == 'all-spark2.4'
Requires-Dist: hmsclient (<1.0.0,>=0.0.1) ; extra == 'all-spark2.4'
Requires-Dist: papermill (>=1.2.0) ; extra == 'all-spark2.4'
Requires-Dist: nbconvert (>=6.0.7) ; extra == 'all-spark2.4'
Requires-Dist: ipykernel (>=5.0.0) ; extra == 'all-spark2.4'
Requires-Dist: black (==19.10b0) ; extra == 'all-spark2.4'
Requires-Dist: sagemaker-training (<4.0.0,>=3.6.2) ; extra == 'all-spark2.4'
Provides-Extra: all-spark3
Requires-Dist: pyspark (>=3.0.0) ; extra == 'all-spark3'
Requires-Dist: k8s-proto (<1.0.0,>=0.0.3) ; extra == 'all-spark3'
Requires-Dist: numpy (<2.0.0,>=1.14.0) ; extra == 'all-spark3'
Requires-Dist: pandas (<2.0.0,>=0.22.0) ; extra == 'all-spark3'
Requires-Dist: pyarrow (<3.0.0,>2.0.0) ; extra == 'all-spark3'
Requires-Dist: hmsclient (<1.0.0,>=0.0.1) ; extra == 'all-spark3'
Requires-Dist: papermill (>=1.2.0) ; extra == 'all-spark3'
Requires-Dist: nbconvert (>=6.0.7) ; extra == 'all-spark3'
Requires-Dist: ipykernel (>=5.0.0) ; extra == 'all-spark3'
Requires-Dist: black (==19.10b0) ; extra == 'all-spark3'
Requires-Dist: sagemaker-training (<4.0.0,>=3.6.2) ; extra == 'all-spark3'
Provides-Extra: hive_sensor
Requires-Dist: hmsclient (<1.0.0,>=0.0.1) ; extra == 'hive_sensor'
Provides-Extra: notebook
Requires-Dist: papermill (>=1.2.0) ; extra == 'notebook'
Requires-Dist: nbconvert (>=6.0.7) ; extra == 'notebook'
Requires-Dist: ipykernel (>=5.0.0) ; extra == 'notebook'
Requires-Dist: black (==19.10b0) ; extra == 'notebook'
Provides-Extra: sagemaker
Requires-Dist: sagemaker-training (<4.0.0,>=3.6.2) ; extra == 'sagemaker'
Provides-Extra: schema
Requires-Dist: numpy (<2.0.0,>=1.14.0) ; extra == 'schema'
Requires-Dist: pandas (<2.0.0,>=0.22.0) ; extra == 'schema'
Requires-Dist: pyarrow (<3.0.0,>2.0.0) ; extra == 'schema'
Provides-Extra: sidecar
Requires-Dist: k8s-proto (<1.0.0,>=0.0.3) ; extra == 'sidecar'
Provides-Extra: spark
Requires-Dist: pyspark (<3.0.0,>=2.4.0) ; extra == 'spark'
Provides-Extra: spark3
Requires-Dist: pyspark (>=3.0.0) ; extra == 'spark3'

# Flytekit

[![PyPI version fury.io](https://badge.fury.io/py/flytekit.svg)](https://pypi.python.org/pypi/flytekit/)
[![PyPI download day](https://img.shields.io/pypi/dd/flytekit.svg)](https://pypi.python.org/pypi/flytekit/)
[![PyPI download month](https://img.shields.io/pypi/dm/flytekit.svg)](https://pypi.python.org/pypi/flytekit/)
[![PyPI format](https://img.shields.io/pypi/format/flytekit.svg)](https://pypi.python.org/pypi/flytekit/)
[![PyPI implementation](https://img.shields.io/pypi/implementation/flytekit.svg)](https://pypi.python.org/pypi/flytekit/)
![Codecov](https://img.shields.io/codecov/c/github/lyft/flytekit?style=plastic)


Python Library for easily authoring, testing, deploying, and interacting with Flyte tasks, workflows, and launch plans. To understand more about flyte refer to,
 - [Flyte homepage](https://flyte.org)
 - [Flyte master repository](https://github.com/lyft/flyte)

## Installation

Flytekit is the core extensible library to author Flyte workflows and tasks and interact with Flyte Backend services. Flyte plugins can be installed separately. 

### Base Installation

```bash
pip install flytekit==0.16.0b1
```

### Simple getting started

```python
from flytekit import task, workflow

@task(cache=True, cache_version="1", retries=3)
def sum(x: int, y: int) -> int:
    return x + y

@task(cache=True, cache_version="1", retries=3)
def square(x: int) -> int:
    return x*x

@workflow
def my_workflow(x: int, y: int) -> int:
    return sum(x=square(x=x),y=square(y=y))
```

### Learn Flytekit by example using
- [Learn flytekit by examples](https://flytecookbook.readthedocs.io/)
- [Flytekit API documentation](http://flytekit.readthedocs.io/)
- [Flyte documentation Hub](http://flytekit.readthedocs.io/)

### Plugins:
Refer to [plugins/README.md](plugins/README.md) for a list of available
plugins. There may be plugins outside of this list, but this list is maintained
by the core maintainers.

## Development
Guide to contribute and develop flytekit

### Recipes

```
$ make
Available recipes:
  setup        Install requirements
  fmt          Format code with black and isort
  lint         Run linters
  test         Run tests
  requirements Compile requirements
```

### Setup (Do Once)

```bash
virtualenv ~/.virtualenvs/flytekit
source ~/.virtualenvs/flytekit/bin/activate
make setup
```

### Formatting

We use [black](https://github.com/psf/black) and [isort](https://github.com/timothycrosley/isort) to autoformat code. Run the following command to execute the formatters:

```bash
source ~/.virtualenvs/flytekit/bin/activate
make fmt
```

### Testing

#### Unit Testing

```bash
source ~/.virtualenvs/flytekit/bin/activate
make test
```

### Updating requirements

Update requirements in [`requirements.in`](requirements.in) (or [`requirements-spark3.in`](requirements-spark3.in)), or update requirements for development in [`dev-requirements.in`](dev-requirements.in). Then, validate, pin and freeze all requirements by running:

```bash
source ~/.virtualenvs/flytekit/bin/activate
make requirements
```

This will re-create the [`requirements.txt`](requirements.txt) (or [`requirements-spark3.in`](requirements-spark3.in)) and [`dev-requirements.txt`](dev-requirements.txt) files which will be used for testing. You will have also have to re-run `make setup` to update your local environment with the updated requirements.


