# -*- coding: utf-8 -*-
from setuptools import setup

packages = \
['indata',
 'indata.dataio',
 'indata.exception',
 'indata.plot',
 'indata.table',
 'indata.utils']

package_data = \
{'': ['*']}

install_requires = \
['numpy>=1.22.3,<2.0.0',
 'pandas>=1.4.2,<2.0.0',
 'plotly>=5.7.0,<6.0.0',
 'tabulate>=0.8.9,<0.9.0']

setup_kwargs = {
    'name': 'indata',
    'version': '1.1.0',
    'description': 'A tool in order to easily generate data quality reports from ABTs, visualize data and manipulate the ABT (analytics base table)',
    'long_description': '[![Apache 2.0 License][license-shield]][license-url]\n![example workflow](https://github.com/RaphSku/indata/actions/workflows/ci.yml/badge.svg)\n[![InData CD](https://github.com/RaphSku/indata/actions/workflows/cd.yml/badge.svg)](https://github.com/RaphSku/indata/actions/workflows/cd.yml)\n\n# indata\nInData is a concise project which enables the user to generate data quality reports with ease. Data exploration and data visualisation tools are also provided for convenience.\n\n### How to install it\n```bash \npip install indata\n```\n\n### How it is organised\n```bash\n.\n└── indata/\n    ├── dataio\n    ├── table\n    ├── plot\n    └── utils\n```\n\n### How to use it\n\n#### dataio\nProvides you with everything you need in order to define the data source\n```python\ndataset = indata.dataio.DataSet("./data.csv")\n```\nand get it loaded\n```python\ndataloader = indata.dataio.DataLoader(dataset)\n```\n\n#### table\nAfter you have defined your data source and constructed out of it a loader, you can start creating a data quality report\n```python\nanalytics_table = indata.table.DataQualityTable(dataloader)\n```\nNow, if you want, you can have a look at the columns and what data type they have\n```python\nanalytics_table.print_header_infos()\n```\nDefine which columns you want to investigate and which columns are continous and which are categorical. With these information, you can kick-off the creation of the data quality report\n```python\ncontinuous_features  = ["Popularity", "Vote_Count", "Vote_Average"]\ncategorical_features = ["Release_Date", "Title", "Overview", "Original_Language", "Genre", "Poster_Url"]\n\ndqt_cont, dqt_catg = analytics_table.create_table(continuous_features = continuous_features,\n                                                  categorical_features = categorical_features,\n                                                  store_json_dir = "./dqt")\n```\nIn the folder `./dqt` you will find two json files, one for the categorical features and one for the continous features. Each file represents a data quality report for the respective group of features.\n\n### Advanced Usage\n#### Transformer\nIf you have looked into what the different packages have to offer, you will notice that the `DataLoader` accepts another optional parameter called `transformer` which is an instance of the `indata.dataio.Transformer` class. A transformer acts on the dataframe and transforms the columns according to a defined transformer function. For instance, you can define the following Transformer\n```python\ntransformer = indata.dataio.Transformer(columns = ["column1"], funcs = indata.dataio.impute_median)\ndataloader  = indata.dataio.DataLoader(dataset)\ndataloader.read_csv(transformer = transformer)\n```\nwhich will impute the missing values with the median.\n\n#### Plotting\nCurrently, there are 3 supported plots: **boxplots**, **distribution plots** and **SPLOMS**. Let\'s see how fast we can create plots out of our data. All you need to get started is a dataframe with some data in it.\n\n##### Boxplots\n```python\nboxplot = indata.plot.BoxPlot(name = "Some nice Boxplot", data = df["column1"], store_dir = "./")\nboxplot.plot()\n```\nThis will create a boxplot for `column1` and will create a directory `plots/boxplots/` at `store_dir` which will hold the `.html` files. These you can open and interactively explore the boxplot.\n\n##### Distribution Plots\nDistribution plots come in two flavours, a distribution plotter for categorical features and one for continous features. The distribution plotter expect two additional parameters, one is the data quality report and a `label_hash`. The data quality report is needed because statistics are extracted from it and plotted inside of the visualisation to enrich it with further details. And `label_hash` is only used for the categorical distribution plotter, you do not need to bother what it does, just use the utility function as you can see below\n```python\ncdist      = indata.plot.ContinuousDistributionPlotter(name = "Some Feature", data = data["feature1"], dqt = cqt,\n                                                       store_dir = "./")\ncdist.plot()\n\nlabel_hash = indata.utils.count.Categories.count(data = df["feature2"]\ncat_dist   = indata.plot.CategoricalDistributionPlotter(name = "Some other Feature", data = df["feature2"], dqt = cat_qt, label_hash = label_hash,\n                                                        store_dir = "./")\ncat_dist.plot()\n```\n\n##### SPLOM\nSPLOM stands for scatterplot matrix and is essentially a matrix of plots where each feature gets plotted against each other. This is useful if you want to investigate linear relationships between the features with just one glance. You can create a SPLOM as simple as running\n```python\nsplom = indata.plot.SPLOM(name = "SPLOM", continuous_data = data[["feature1", "feature2", "feature3"]], \n                          store_dir = "./")\n```\n\n### Results\nIn this case, I want to show some results which I got when using this library on a movie dataset which contains different movie titles and their popularity.\n\nWe get the following data quality report for the continuous features of that dataset:\n```json\n{\n    "Count": {\n        "Popularity": 9827,\n        "Vote_Count": 9827,\n        "Vote_Average": 9827\n    },\n    "Miss. %": {\n        "Popularity": 0.1016570093,\n        "Vote_Count": 0.1016570093,\n        "Vote_Average": 0.1016570093\n    },\n    "Card.": {\n        "Popularity": 8160,\n        "Vote_Count": 3267,\n        "Vote_Average": 75\n    },\n    "Min": {\n        "Popularity": 7.1,\n        "Vote_Count": 7.1,\n        "Vote_Average": 7.1\n    },\n    "1st Qrt.": {\n        "Popularity": 16.1275,\n        "Vote_Count": 16.1275,\n        "Vote_Average": 16.1275\n    },\n    "mean": {\n        "Popularity": 40.3205699603,\n        "Vote_Count": 40.3205699603,\n        "Vote_Average": 40.3205699603\n    },\n    "median": {\n        "Popularity": 21.191,\n        "Vote_Count": 21.191,\n        "Vote_Average": 21.191\n    },\n    "3rd Qrt.": {\n        "Popularity": 35.1745,\n        "Vote_Count": 35.1745,\n        "Vote_Average": 35.1745\n    },\n    "Max": {\n        "Popularity": 5083.954,\n        "Vote_Count": 5083.954,\n        "Vote_Average": 5083.954\n    },\n    "Std. Dev.": {\n        "Popularity": 108.8743077303,\n        "Vote_Count": 108.8743077303,\n        "Vote_Average": 108.8743077303\n    }\n}\n```\n\nAnd a data quality report for the categorical features:\n```json\n{\n    "Count": {\n        "Release_Date": 9837,\n        "Title": 9828,\n        "Overview": 9828,\n        "Original_Language": 9827,\n        "Genre": 9826,\n        "Poster_Url": 9826\n    },\n    "Miss. %": {\n        "Release_Date": 0.0,\n        "Title": 0.0914913083,\n        "Overview": 0.0914913083,\n        "Original_Language": 0.1016570093,\n        "Genre": 0.1118227102,\n        "Poster_Url": 0.1118227102\n    },\n    "Card.": {\n        "Release_Date": 5903,\n        "Title": 9514,\n        "Overview": 9823,\n        "Original_Language": 44,\n        "Genre": 2337,\n        "Poster_Url": 9826\n    },\n    "Mode": {\n        "Release_Date": "2022-03-10",\n        "Title": "Beauty and the Beast",\n        "Overview": "Dr. Raichi is one of the only survivors of the Tuffles, a race that once lived on Planet Plant before the coming of the Saiyans. The Saiyans not only massacred the entire Tuffle race, but also stole their technology and conquered the planet, renaming it Planet Vegeta in honor of their king. Raichi managed to escape with a capsule and found refuge on the Dark Planet, a world at the end of the universe. His only wish is to eradicate the last remaining Saiyans.",\n        "Original_Language": "en",\n        "Genre": "Drama",\n        "Poster_Url": "https:\\/\\/image.tmdb.org\\/t\\/p\\/original\\/1g0dhYtq4irTY1GPXvft6k4YLjm.jpg"\n    },\n    "Mode Freq.": {\n        "Release_Date": 16,\n        "Title": 4,\n        "Overview": 2,\n        "Original_Language": 7569,\n        "Genre": 466,\n        "Poster_Url": 1\n    },\n    "Mode Freq. %": {\n        "Release_Date": 0.1626512148,\n        "Title": 0.0407000407,\n        "Overview": 0.0203500204,\n        "Original_Language": 77.0224890608,\n        "Genre": 4.7425198453,\n        "Poster_Url": 0.0101770812\n    },\n    "2nd Mode": {\n        "Release_Date": "2022-03-09",\n        "Title": "Alice in Wonderland",\n        "Overview": "Wilbur the pig is scared of the end of the season, because he knows that come that time, he will end up on the dinner table. He hatches a plan with Charlotte, a spider that lives in his pen, to ensure that this will never happen.",\n        "Original_Language": "ja",\n        "Genre": "Comedy",\n        "Poster_Url": "https:\\/\\/image.tmdb.org\\/t\\/p\\/original\\/deOzvJHnSSl8FI1HEJjPGgOsS9U.jpg"\n    },\n    "2nd Mode Freq.": {\n        "Release_Date": 15,\n        "Title": 4,\n        "Overview": 2,\n        "Original_Language": 645,\n        "Genre": 403,\n        "Poster_Url": 1\n    },\n    "2nd Mode Freq. %": {\n        "Release_Date": 0.1524855139,\n        "Title": 0.0407000407,\n        "Overview": 0.0203500204,\n        "Original_Language": 6.5635494047,\n        "Genre": 4.1013637289,\n        "Poster_Url": 0.0101770812\n    }\n}\n```\n\n\n  \n[contributors-url]: https://github.com/RaphSku\n[license-url]: https://github.com/RaphSku/indata/blob/main/LICENSE\n\n[license-shield]: https://img.shields.io/badge/License-Apache%202.0-orange\n',
    'author': 'RapSku',
    'author_email': 'rapsku.dev@gmail.com',
    'maintainer': None,
    'maintainer_email': None,
    'url': 'https://github.com/RaphSku/indata',
    'packages': packages,
    'package_data': package_data,
    'install_requires': install_requires,
    'python_requires': '>=3.9,<4.0',
}


setup(**setup_kwargs)
