# -*- coding: utf-8 -*-
from setuptools import setup

packages = \
['whylogs',
 'whylogs.api',
 'whylogs.api.logger',
 'whylogs.api.pyspark.experimental',
 'whylogs.api.usage_stats',
 'whylogs.api.writer',
 'whylogs.core',
 'whylogs.core.constraints',
 'whylogs.core.metrics',
 'whylogs.core.proto',
 'whylogs.core.proto.v0',
 'whylogs.core.utils',
 'whylogs.core.view',
 'whylogs.migration',
 'whylogs.viz',
 'whylogs.viz.utils']

package_data = \
{'': ['*'],
 'whylogs.viz': ['html/*',
                 'html/css/*',
                 'html/fonts/*',
                 'html/images/*',
                 'html/js/*',
                 'html/templates/*']}

install_requires = \
['protobuf>=3.15.5', 'typing-extensions', 'whylogs-sketching==3.4.0.dev9']

extras_require = \
{':python_version < "3.8"': ['importlib-metadata>=0.23'],
 'docs': ['sphinx',
          'sphinx-autoapi',
          'sphinx-copybutton>=0.5.0,<0.6.0',
          'myst-parser[sphinx]>=0.17.2,<0.18.0',
          'furo>=2022.3.4,<2023.0.0',
          'sphinx-autobuild>=2021.3.14,<2022.0.0',
          'sphinxext-opengraph>=0.6.3,<0.7.0',
          'sphinx-inline-tabs',
          'nbsphinx>=0.8.8,<0.9.0'],
 's3': ['boto3>=1.22.13,<2.0.0'],
 'spark': ['pyarrow>=8.0.0,<9.0.0', 'pyspark>=3.0.0,<4.0.0'],
 'viz': ['pybars3>=0.9,<0.10', 'ipython', 'scipy<1.8'],
 'whylabs': ['whylabs-client>=0.3.0,<0.4.0']}

setup_kwargs = {
    'name': 'whylogs',
    'version': '1.0.0rc1',
    'description': 'Profile and monitor your ML data pipeline end-to-end',
    'long_description': '<img src="https://static.scarf.sh/a.png?x-pxid=bc3c57b0-9a65-49fe-b8ea-f711c4d35b82" /><p align="center">\n<img src="https://i.imgur.com/nv33goV.png" width="35%"/>\n</br>\n\n<h1 align="center">The open source standard for data logging\n </h1>\n  <h3 align="center">\n   <a href="https://docs.whylabs.ai/docs/"><b>Documentation</b></a> &bull;\n   <a href="https://bit.ly/whylogsslack"><b>Slack Community</b></a> &bull;\n   <a href="https://github.com/whylabs/whylogs#python-quickstart"><b>Python Quickstart</b></a>\n </h3>\n\n<p align="center">\n<a href="https://github.com/whylabs/whylogs-python/blob/mainline/LICENSE" target="_blank">\n    <img src="http://img.shields.io/:license-Apache%202-blue.svg" alt="License">\n</a>\n<a href="https://badge.fury.io/py/whylogs" target="_blank">\n    <img src="https://badge.fury.io/py/whylogs.svg" alt="PyPi Version">\n</a>\n<a href="https://github.com/python/black" target="_blank">\n    <img src="https://img.shields.io/badge/code%20style-black-000000.svg" alt="Code style: black">\n</a>\n<a href="https://pepy.tech/project/whylogs" target="_blank">\n    <img src="https://pepy.tech/badge/whylogs" alt="PyPi Downloads">\n</a>\n<a href="bit.ly/whylogs" target="_blank">\n    <img src="https://github.com/whylabs/whylogs-python/workflows/whylogs%20CI/badge.svg" alt="CI">\n</a>\n<a href="https://codeclimate.com/github/whylabs/whylogs-python/maintainability" target="_blank">\n    <img src="https://api.codeclimate.com/v1/badges/442f6ca3dca1e583a488/maintainability" alt="Maintainability">\n</a>\n</p>\n\n## What is whylogs\n\nwhylogs is an open source library for logging any kind of data. With whylogs, users are able to generate summaries of\ntheir datasets (called _whylogs profiles_) which they can use to:\n\n1. Track changes in their dataset\n2. Create _data constraints_ to know whether their data looks they way it should\n3. Quickly visualize key summary statistics about their datasets\n\nThese three functionalities enable a variety of use cases for data scientists, machine learning engineers, and data\nengineers:\n\n- Detecting data drift (and resultant ML model performance degradation)\n- Data quality validation\n- Exploratory data analysis via data profiling\n- Tracking data for ML experiments\n- And many more\n\nwhylogs can be run in Python or [Apache Spark](https://docs.whylabs.ai/docs/spark-integration) (both PySpark and Scala)\nenvironments on a variety of [data types](#data-types). We [integrate](#integrations) with lots of other tools including\nPandas, [AWS Sagemaker](https://aws.amazon.com/blogs/startups/preventing-amazon-sagemaker-model-degradation-with-whylabs/)\n, [MLflow](https://docs.whylabs.ai/docs/mlflow-integration)\n, [Flask](https://whylabs.ai/blog/posts/deploy-and-monitor-your-ml-application-with-flask-and-whylabs)\n, [Ray](https://docs.whylabs.ai/docs/ray-integration)\n, [RAPIDS](https://whylabs.ai/blog/posts/monitoring-high-performance-machine-learning-models-with-rapids-and-whylogs)\n, [Apache Kafka](https://docs.whylabs.ai/docs/kafka-integration), and more.\n\nIf you have any questions, comments, or just want to hang out with us, please\njoin [our Slack Community](http://join.slack.whylabs.ai/). In addition to joining the Slack Community, you can also help\nthis project by giving us a ⭐ in the upper right corner of this page.\n\n## Python Quickstart<a name="python-quickstart" />\n\n### Install whylogs\n\nInstall whylogs using the pip package manager by running:\n\n```\npip install whylogs\n```\n\n### Log some data\n\nProfiling dataframes, Python dicts, or images is as easy as running:\n\n```python\nfrom whylogs import get_or_create_session\nimport pandas as pd\n\nsession = get_or_create_session()\n\nwith session.logger(dataset_name="my_dataset") as logger:\n    logger.log_dataframe(df)\n```\n\n## Table of Contents\n\n- [whylogs Profiles](#whylogs-profiles)\n- [Visualizing Profiles](#visualizing-profiles)\n- [Features](#features)\n- [Data Types](#data-types)\n- [Integrations](#integrations)\n- [Examples](#examples)\n- [Community](#community)\n- [Roadmap](#roadmap)\n- [Contribute](#contribute)\n\n## whylogs Profiles<a name="whylogs-profiles" />\n\n### What are profiles\n\nwhylogs profiles are the core of the whylogs library. They capture key statistical properties of data, such as the\ndistribution (far beyond simple mean, median, and standard deviation measures), the number of missing values, and a wide\nrange of configurable custom metrics. By capturing these summary statistics, we are able to accurately represent the\ndata and enable all of the use cases described in the introduction.\n\nwhylogs profiles have three properties that make them ideal for data logging: they are **descriptive**, **lightweight**,\nand **mergeable**.\n\n<br />\n\n<img align="left" src="https://user-images.githubusercontent.com/7946482/156462388-ceb4cabd-8a5d-4885-8889-fe393a7b2527.png">\n\n**Descriptive**: whylogs profiles describe the dataset that they represent. This high fidelity representation of\ndatasets is what enables whylogs profiles to be effective snapshots of the data. They are better at capturing the\ncharacteristics of a dataset than a sample would be, as discussed in\nour [Data Logging: Sampling versus Profiling](https://whylabs.ai/blog/posts/data-logging-sampling-versus-profiling) blog\npost.\n\n<br />\n\n<img align="left" src="https://user-images.githubusercontent.com/7946482/156462585-6d679747-c9c6-4724-8640-d85fc70794ca.png">\n\n**Lightweight**: In addition to being a high fidelity representation of data, whylogs profiles also have high\ninformation density. You can easily profile terabytes or even petabytes of data in profiles that are only megabytes\nlarge. Because whylogs profiles are lightweight, they are very inexpensive to store, transport, and interact with.\n\n<br />\n\n<img align="left" src="https://user-images.githubusercontent.com/7946482/156462686-5320701b-dbb5-4981-af7f-28f8c25e6738.png">\n\n**Mergeable**: One of the most powerful features of whylogs profiles is their mergeability. Mergeability means that\nwhylogs profiles can be combined together to form new profiles which represent the aggregate of their constituent\nprofiles. This enables logging for distributed and streaming systems, and allows users to view aggregated data across\nany time granularity.\n\n### How do you generate profiles\n\nOnce whylogs is installed, it\'s easy to generate profiles in both Python and Java environments.\n\nTo generate a profile from a Pandas dataframe in Python, simply run:\n\n```python\nfrom whylogs import get_or_create_session\nimport pandas as pd\n\nsession = get_or_create_session()\n\ndf = pd.read_csv("path/to/file.csv")\n\nwith session.logger(dataset_name="my_dataset") as logger:\n    logger.log_dataframe(df)\n```\n\nFor images, replace `logger.log_dataframe(df)` with `logger.log_image("path/to/image.png")`. Similarly, you can profile\nPython dicts by running `logger.log(dict)` directly on the dict object.\n\n### What can you do with profiles\n\nOnce you’ve generated whylogs profiles, a few things can be done with them:\n\nIn your local Python environment, you can set data constraints or visualize your profiles. Setting data constraints on\nyour profiles allows you to get notified when your data don’t match your expectations, allowing you to do data unit\ntesting and some baseline data monitoring. With the Profile Visualizer, you can visually explore your data, allowing you\nto understand it and ensure that your ML models are ready for production.\n\nIn addition, you can send whylogs profiles to the SaaS ML monitoring and AI observability platform [WhyLabs](whylabs.ai)\n. With WhyLabs, you can automatically set up monitoring for your machine learning models, getting notified on both data\nquality and data change issues (such as data drift). If you’re interested in trying out WhyLabs, check out the always\nfree [Starter edition](whylabs.ai/free), which allows you to experience the entire platform’s capabilities with no\ncredit card required.\n\n## Visualizing Profiles<a name="visualizing-profiles" />\n\n### Multiple profile plots\n\nTo view your logger profiles you can use, methods within `whylogs.viz`:\n\n```python\nvizualization = ProfileVisualizer()\nvizualization.set_profiles([profile_day_1, profile_day_2])\nfigure = vizualization.plot_distribution("<feature_name>")\nfigure.savefig("/my/image/path.png")\n```\n\nIndividual profiles can be saved to local disk, uploaded to AWS S3, or sent to the WhyLabs AI Observability Platform via\nthe WhyLabs API. This can be set up to happen automatically when loggers are closed, per the configuration found in the\nSession configuration.\n\nCurrent profiles from active loggers can be loaded from memory with:\n\n```python\nprofile = logger.profile()\n```\n\n### Profile viewer\n\nYou can also load a local profile viewer, where you upload the `json` summary file. The default path for the json files\nis set as `output/{dataset_name}/{session_id}/json/dataset_profile.json`.\n\n```python\nfrom whylogs.viz import profile_viewer\n\nprofile_viewer()\n```\n\nThis will open a viewer on your default browser where you can load a profile json summary, using\nthe `Select JSON profile` button:\nOnce the json is selected you can view your profile\'s features and associated statistics.\n\n<img src="https://whylabs-public.s3-us-west-2.amazonaws.com/assets/whylogs-viewer.gif" title="whylogs HTML viewer demo">\n\n## Features\n\nwhylogs collects approximate statistics and sketches of data on a column-basis into a statistical profile. These metrics\ninclude:\n\n- Simple counters: boolean, null values, data types.\n- Summary statistics: sum, min, max, median, variance.\n- Unique value counter or cardinality: tracks an approximate unique value of your feature using HyperLogLog algorithm.\n- Histograms for numerical features. whyLogs binary output can be queried to with dynamic binning based on the shape of\n  your data.\n- Top frequent items (default is 128). Note that this configuration affects the memory footprint, especially for text\n  features.\n\nSome other key features about whylogs:\n\n- Accurate data profiling: whylogs calculates statistics from 100% of the data, never requiring sampling, ensuring an\n  accurate representation of data distributions\n- Lightweight runtime: whylogs utilizes approximate statistical methods to achieve minimal memory footprint that scales\n  with the number of features in the data\n- Any architecture: whylogs scales with your system, from local development mode to live production systems in\n  multi-node clusters, and works well with batch and streaming architectures\n- Configuration-free: whylogs infers the schema of the data, requiring zero manual configuration to get started\n- Tiny storage footprint: whylogs turns data batches and streams into statistical fingerprints, 10-100MB uncompressed\n- Unlimited metrics: whylogs collects all possible statistical metrics about structured or unstructured data\n\n## Data Types<a name="data-types" />\n\nwhylogs supports both structured and unstructured data, specifically:\n\n| Data type       | Features                                                  | Notebook Example                                                                                                              |\n| --------------- | --------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- |\n| Structured Data | Distribution, cardinality, schema, counts, missing values | [Getting started with structured data](https://github.com/whylabs/whylogs-examples/blob/mainline/python/GettingStarted.ipynb) |\n| Images          | exif metadata, derived pixels features, bounding boxes    | [Getting started with images](https://github.com/whylabs/whylogs-examples/blob/mainline/python/Logging_Images.ipynb)          |\n| Video           | In development                                            | [Github Issue #214](https://github.com/whylabs/whylogs/issues/214)                                                            |\n| Tensors         | derived 1d features (more in developement)                | [Github Issue #216](https://github.com/whylabs/whylogs/issues/216)                                                            |\n| Text            | top k values, counts, cardinality                         | [String Features](https://github.com/whylabs/whylogs/blob/mainline/examples/String_Features.ipynb)                            |\n| Audio           | In developement                                           | [Github Issue #212](https://github.com/whylabs/whylogs/issues/212)                                                            |\n\n## Integrations\n\n![current integration](images/integrations.001.png)\n| Integration | Features | Resources | | --- | --- | --- | | Spark | Run whylogs in Apache Spark\nenvironment| <ul><li>[Code Example](https://github.com/whylabs/whylogs-examples/blob/mainline/scala/src/main/scala/WhyLogsDemo.scala)</li></ul>\n| | Pandas | Log and monitor any pandas dataframe\n| <ul><li>[Notebook Example](https://github.com/whylabs/whylogs-examples/blob/mainline/python/logging_example.ipynb)</li><li>[whylogs:\nEmbrace Data Logging](https://whylabs.ai/blog/posts/whylogs-embrace-data-logging)</li></ul> | | Kafka | Log and monitor\nKafka topics with\nwhylogs| <ul><li>[Notebook Example](https://github.com/whylabs/whylogs-examples/blob/mainline/python/Kafka.ipynb)</li><li> [Integrating whylogs into your Kafka ML Pipeline](https://whylabs.ai/blog/posts/integrating-whylogs-into-your-kafka-ml-pipeline) </li></ul>\n| | MLflow | Enhance MLflow metrics with whylogs:\n| <ul><li>[Notebook Example](https://github.com/whylabs/whylogs-examples/blob/mainline/python/MLFlow%20Integration%20Example.ipynb)</li><li>[Streamlining\ndata monitoring with whylogs and MLflow](https://whylabs.ai/blog/posts/on-model-lifecycle-and-monitoring)</li></ul> | |\nGithub actions | Unit test data with whylogs and github\nactions| <ul><li>[Notebook Example](https://github.com/whylabs/whylogs-examples/tree/mainline/github-actions)</li></ul>\n| | RAPIDS | Use whylogs in RAPIDS environment\n| <ul><li>[Notebook Example](https://github.com/whylabs/whylogs-examples/blob/mainline/python/RAPIDS%20GPU%20Integration%20Example.ipynb)</li><li>[Monitoring\nHigh-Performance Machine Learning Models with RAPIDS and\nwhylogs](https://whylabs.ai/blog/posts/monitoring-high-performance-machine-learning-models-with-rapids-and-whylogs)</li></ul>\n| | Java | Run whylogs in Java\nenvironment| <ul><li>[Notebook Example](https://github.com/whylabs/whylogs-examples/blob/mainline/java/demo1/src/main/java/com/whylogs/examples/WhyLogsDemo.java)</li></ul>\n| | Docker | Run whylogs as in Docker\n| <ul><li>[Rest Container](https://docs.whylabs.ai/docs/integrations-rest-container)</li></ul>|\n| AWS S3 | Store whylogs profiles in S3\n| <ul><li>[S3 example](https://github.com/whylabs/whylogs-examples/blob/mainline/python/S3%20example.ipynb)</li></ul>\n\n## Examples\n\nFor a full set of our examples, please check out the [examples folder](/examples/).\n\nCheck out our example notebooks with\nBinder: [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/whylabs/whylogs-examples/HEAD)\n\n- [Getting Started notebook](https://github.com/whylabs/whylogs-examples/blob/mainline/python/GettingStarted.ipynb)\n- [Logging Example notebook](https://github.com/whylabs/whylogs-examples/blob/mainline/python/logging_example.ipynb)\n- [Logging Images](https://github.com/whylabs/whylogs-examples/blob/mainline/python/Logging_Images.ipynb)\n- [MLflow Integration](https://github.com/whylabs/whylogs-examples/blob/mainline/python/MLFlow%20Integration%20Example.ipynb)\n\n## Roadmap\n\nwhylogs is maintained by [WhyLabs](https://whylabs.ai).\n\n## Community\n\nIf you have any questions, comments, or just want to hang out with us, please\njoin [our Slack channel](http://join.slack.whylabs.ai/).\n\n## Contribute\n\n### How to Contribute\n\nWe welcome contributions to whylogs. Please see\nour [contribution guide](https://github.com/whylabs/whylogs/blob/mainline/CONTRIBUTING.md) and\nour [development guide](https://github.com/whylabs/whylogs/blob/mainline/DEVELOPMENT.md) for details.\n\n### Contributors\n\n<a href="https://github.com/whylabs/whylogs/graphs/contributors">\n  <img src="https://contrib.rocks/image?repo=whylabs/whylogs" />\n</a>\n\nMade with [contrib.rocks](https://contrib.rocks).\n',
    'author': 'WhyLabs.ai',
    'author_email': 'support@whylabs.ai',
    'maintainer': None,
    'maintainer_email': None,
    'url': 'https://docs.whylabs.ai',
    'packages': packages,
    'package_data': package_data,
    'install_requires': install_requires,
    'extras_require': extras_require,
    'python_requires': '>=3.7.1,<4.0',
}


setup(**setup_kwargs)
