# -*- coding: utf-8 -*-
from setuptools import setup

packages = \
['qabot', 'qabot.agents', 'qabot.tools']

package_data = \
{'': ['*']}

install_requires = \
['duckdb-engine>=0.6.9,<0.7.0',
 'langchain>=0.0.107,<0.0.108',
 'openai>=0.27.0,<0.28.0',
 'psycopg2-binary>=2.9.5,<3.0.0',
 'rich>=13.3.1,<14.0.0',
 'typer>=0.7.0,<0.8.0']

entry_points = \
{'console_scripts': ['qabot = qabot.cli:run']}

setup_kwargs = {
    'name': 'qabot',
    'version': '0.2.7',
    'description': 'Query local or remote data files with natural language queries powered by OpenAI and DuckDB.',
    'long_description': '# qabot\n\nQuery local or remote files with natural language queries powered by\n`langchain` and `gpt-3.5-turbo` and `duckdb` 🦆.\n\n\n\nWorks on local CSV files:\n\n![](.github/local_csv_query.png)\n\nremote CSV files:\n\n```\n$ qabot \\\n    -f https://www.stats.govt.nz/assets/Uploads/Environmental-economic-accounts/Environmental-economic-accounts-data-to-2020/renewable-energy-stock-account-2007-2020-csv.csv \\\n    -q "How many Gigawatt hours of generation was there for Solar resources in 2015 through to 2020?"\n```\n\n\nEven on (public) data stored in S3:\n\n![](.github/external_s3_data.png)\n\nYou can even load data from disk via the natural language query, but that doesn\'t always work...\n\n\n> "Load the file \'data/titanic_survival.parquet\' into a table called \'raw_passengers\'. Create a view of the raw passengers table for just the male passengers. What was the average fare for surviving male passengers?"\n\n\nAfter a bit of back and forth with the model, it gets there:\n\n> The average fare for surviving male passengers from the \'male_passengers\' view where the passenger survived is 40.82. I ran the query: SELECT AVG(Fare) FROM male_passengers WHERE Survived = 1 AND Sex = \'male\';\nThe average fare for surviving male passengers is 40.82.\n\n\n## Quickstart\n\nYou need to set the `OPENAI_API_KEY` environment variable to your OpenAI API key, \nwhich you can get from [here](https://platform.openai.com/account/api-keys).\n\nInstall the `qabot` command line tool using pip/poetry:\n\n\n```bash\n$ pip install qabot\n```\n\nThen run the `qabot` command with either local files (`-f my-file.csv`) or a database connection string.\n\nNote if you want to use a database, you will need to install the relevant drivers, \ne.g. `pip install psycopg2-binary` for postgres.\n\n\n\n## Examples\n\n### Local CSV file/s\n\n```bash\n$ qabot -q "how many passengers survived by gender?" -f data/titanic.csv\n🦆 Loading data from files...\nLoading data/titanic.csv into table titanic...\n\nQuery: how many passengers survived by gender?\nResult:\nThere were 233 female passengers and 109 male passengers who survived.\n\n\n 🚀 any further questions? [y/n] (y): y\n\n 🚀 Query: what was the largest family who did not survive? \nQuery: what was the largest family who did not survive?\nResult:\nThe largest family who did not survive was the Sage family, with 8 members.\n\n 🚀 any further questions? [y/n] (y): n\n```\n\n\n## Intermediate steps and database queries\n\nUse the `-v` flag to see the intermediate steps and database queries.\n\nSometimes it takes a long route to get to the answer, but it\'s interesting to see how it gets there:\n\n\n```\nqabot -f data/titanic.csv -q "how many passengers survived by gender?" -v\n🦆 Loading data from files...\nQuery: how many passengers survived by gender?\nI need to check the columns in the \'titanic\' table to see which ones contain gender and survival information.\nAction: Describe Table\nAction Input: titanic\n\nObservation: titanic\n\n┌─────────────┬─────────────┬─────────┬─────────┬─────────┬───────┐\n│ column_name │ column_type │  null   │   key   │ default │ extra │\n│   varchar   │   varchar   │ varchar │ varchar │ varchar │ int32 │\n├─────────────┼─────────────┼─────────┼─────────┼─────────┼───────┤\n│ PassengerId │ BIGINT      │ YES     │ NULL    │ NULL    │  NULL │\n│ Survived    │ BIGINT      │ YES     │ NULL    │ NULL    │  NULL │\n│ Pclass      │ BIGINT      │ YES     │ NULL    │ NULL    │  NULL │\n│ Name        │ VARCHAR     │ YES     │ NULL    │ NULL    │  NULL │\n│ Sex         │ VARCHAR     │ YES     │ NULL    │ NULL    │  NULL │\n│ Age         │ DOUBLE      │ YES     │ NULL    │ NULL    │  NULL │\n│ SibSp       │ BIGINT      │ YES     │ NULL    │ NULL    │  NULL │\n│ Parch       │ BIGINT      │ YES     │ NULL    │ NULL    │  NULL │\n│ Ticket      │ VARCHAR     │ YES     │ NULL    │ NULL    │  NULL │\n│ Fare        │ DOUBLE      │ YES     │ NULL    │ NULL    │  NULL │\n│ Cabin       │ VARCHAR     │ YES     │ NULL    │ NULL    │  NULL │\n│ Embarked    │ VARCHAR     │ YES     │ NULL    │ NULL    │  NULL │\n├─────────────┴─────────────┴─────────┴─────────┴─────────┴───────┤\n│ 12 rows                                               6 columns │\n└─────────────────────────────────────────────────────────────────┘\n\nI need to create a view that only includes the columns I need for this question.\nAction: Data Op\nAction Input: \n        CREATE VIEW titanic_gender_survival AS\n        SELECT Sex, Survived\n        FROM titanic\nThought:\n\n> Entering new AgentExecutor chain...\nThis is a valid SQL query creating a view. We can execute it directly.\nAction: execute\nAction Input: \n        CREATE VIEW titanic_gender_survival AS\n        SELECT Sex, Survived\n        FROM titanic\nObservation: No output\nThought:The view has been created successfully. We can now query it.\nAction: execute\nAction Input: SELECT * FROM titanic_gender_survival LIMIT 5\nObservation: \n┌─────────┬──────────┐\n│   Sex   │ Survived │\n│ varchar │  int64   │\n├─────────┼──────────┤\n│ male    │        0 │\n│ female  │        1 │\n│ female  │        1 │\n│ female  │        1 │\n│ male    │        0 │\n└─────────┴──────────┘\n\nThought:The view has been created successfully and we can see the first 5 rows of the view. The final answer is the first 5 rows of the titanic_gender_survival view, showing the sex and survival status of passengers on the \nTitanic.\nFinal Answer: \n┌─────────┬──────────┐\n│   Sex   │ Survived │\n│ varchar │  int64   │\n├─────────┼──────────┤\n│ male    │        0 │\n│ female  │        1 │\n│ female  │        1 │\n│ female  │        1 │\n│ male    │        0 │\n└─────────┴──────────┘\n\n> Finished chain.\n\nObservation: ┌─────────┬──────────┐\n│   Sex   │ Survived │\n│ varchar │  int64   │\n├─────────┼──────────┤\n│ male    │        0 │\n│ female  │        1 │\n│ female  │        1 │\n│ female  │        1 │\n│ male    │        0 │\n└─────────┴──────────┘\nI need to group the data by gender and count the number of survivors for each group.\nAction: Data Op\nAction Input: \n        SELECT Sex, SUM(Survived) AS num_survived\n        FROM titanic_gender_survival\n        GROUP BY Sex\nThought:\n\n> Entering new AgentExecutor chain...\nWe need to check if the table titanic_gender_survival exists and if it has the columns Sex and Survived.\nAction: Describe Table\nAction Input: titanic_gender_survival\nObservation: titanic_gender_survival\n\n┌─────────────┬─────────────┬─────────┬─────────┬─────────┬───────┐\n│ column_name │ column_type │  null   │   key   │ default │ extra │\n│   varchar   │   varchar   │ varchar │ varchar │ varchar │ int32 │\n├─────────────┼─────────────┼─────────┼─────────┼─────────┼───────┤\n│ Sex         │ VARCHAR     │ YES     │ NULL    │ NULL    │  NULL │\n│ Survived    │ BIGINT      │ YES     │ NULL    │ NULL    │  NULL │\n└─────────────┴─────────────┴─────────┴─────────┴─────────┴───────┘\n\nThought:The table titanic_gender_survival exists and has the columns Sex and Survived. We can now run the query.\nAction: execute\nAction Input: \n```\nSELECT Sex, SUM(Survived) AS num_survived\nFROM titanic_gender_survival\nGROUP BY Sex\nLIMIT 5\n```\n\nObservation: \n┌─────────┬──────────────┐\n│   Sex   │ num_survived │\n│ varchar │    int128    │\n├─────────┼──────────────┤\n│ male    │          109 │\n│ female  │          233 │\n└─────────┴──────────────┘\n\nThought:The query returned the number of survivors grouped by gender. The table titanic_gender_survival has been used. \nFinal Answer: The number of survivors grouped by gender are: \n- 109 males survived\n- 233 females survived.\n\n> Finished chain.\n\nObservation: The number of survivors grouped by gender are: \n- 109 males survived\n- 233 females survived.\nIntermediate Steps: \n  Step 1\n\n    Describe Table\n      titanic\n\n      titanic\n\n    ┌─────────────┬─────────────┬─────────┬─────────┬─────────┬───────┐\n    │ column_name │ column_type │  null   │   key   │ default │ extra │\n    │   varchar   │   varchar   │ varchar │ varchar │ varchar │ int32 │\n    ├─────────────┼─────────────┼─────────┼─────────┼─────────┼───────┤\n    │ PassengerId │ BIGINT      │ YES     │ NULL    │ NULL    │  NULL │\n    │ Survived    │ BIGINT      │ YES     │ NULL    │ NULL    │  NULL │\n    │ Pclass      │ BIGINT      │ YES     │ NULL    │ NULL    │  NULL │\n    │ Name        │ VARCHAR     │ YES     │ NULL    │ NULL    │  NULL │\n    │ Sex         │ VARCHAR     │ YES     │ NULL    │ NULL    │  NULL │\n    │ Age         │ DOUBLE      │ YES     │ NULL    │ NULL    │  NULL │\n    │ SibSp       │ BIGINT      │ YES     │ NULL    │ NULL    │  NULL │\n    │ Parch       │ BIGINT      │ YES     │ NULL    │ NULL    │  NULL │\n    │ Ticket      │ VARCHAR     │ YES     │ NULL    │ NULL    │  NULL │\n    │ Fare        │ DOUBLE      │ YES     │ NULL    │ NULL    │  NULL │\n    │ Cabin       │ VARCHAR     │ YES     │ NULL    │ NULL    │  NULL │\n    │ Embarked    │ VARCHAR     │ YES     │ NULL    │ NULL    │  NULL │\n    ├─────────────┴─────────────┴─────────┴─────────┴─────────┴───────┤\n    │ 12 rows                                               6 columns │\n    └─────────────────────────────────────────────────────────────────┘\n\n    \n\n  Step 2\n\n    Data Op\n      CREATE VIEW titanic_gender_survival AS\n            SELECT Sex, Survived\n            FROM titanic\n\n      ┌─────────┬──────────┐\n    │   Sex   │ Survived │\n    │ varchar │  int64   │\n    ├─────────┼──────────┤\n    │ male    │        0 │\n    │ female  │        1 │\n    │ female  │        1 │\n    │ female  │        1 │\n    │ male    │        0 │\n    └─────────┴──────────┘\n\n    \n\n  Step 3\n\n    Data Op\n      SELECT Sex, SUM(Survived) AS num_survived\n            FROM titanic_gender_survival\n            GROUP BY Sex\n\n      The number of survivors grouped by gender are: \n    - 109 males survived\n    - 233 females survived.\n\n    \n\n\nThought:\n\n\nResult:\n109 males and 233 females survived.\n```\n\n## Data accessed via http/s3\n\nUse the `-f <url>` flag to load data from a url, e.g. a csv file on s3:\n\n```bash\n$ qabot -f s3://covid19-lake/enigma-jhu-timeseries/csv/jhu_csse_covid_19_timeseries_merged.csv -q "how many confirmed cases are there" -v\n🦆 Loading data from files...\ncreate table jhu_csse_covid_19_timeseries_merged as select * from \'s3://covid19-lake/enigma-jhu-timeseries/csv/jhu_csse_covid_19_timeseries_merged.csv\';\n\nResult:\n264308334 confirmed cases\n```\n\n## Links\n\n- [Python library docs](https://langchain.readthedocs.io)\n- [Agent docs to talk to arbitrary apis via OpenAPI/Swagger](https://langchain.readthedocs.io/en/latest/modules/agents/agent_toolkits/openapi.html)\n- [Agents/Tools to talk SQL](https://langchain.readthedocs.io/en/latest/modules/agents/agent_toolkits/sql_database.html)\n- [Typescript library](https://hwchase17.github.io/langchainjs/docs/overview/)\n\n\n\n## Ideas\n\n- Upgrade to use langchain chat interface\n- Use memory, perhaps wait for langchain\'s next release\n- Decent Python Library API so can be used from other Python code\n- streaming mode to output results as they come in\n- token usage reporting and limits\n- Supervisor agent - assess whether a query is "safe" to run, could ask for user confirmation to run anything that gets flagged.\n- Often we can zero-shot the question and get a single query out - perhaps we try this before the MKL chain\n- test each zeroshot agent individually\n- Pass in the available tables from the first prompt\n- Generate and pass back assumptions made to the user\n- Add an optional "clarify" tool to the chain that asks the user to clarify the question\n- Create a query checker tool that checks if the query looks valid\n- Perhaps an explain query tool that shows the steps taken to get the answer\n- Store all queries, actions, and answers in a table\n- Optional settings to switch to different LLM\n- Inject AWS credentials into duckdb so we can access private resources in S3\n- A version that uses document embeddings - probably not in this app as needs Torch',
    'author': 'Brian Thorne',
    'author_email': 'brian@hardbyte.nz',
    'maintainer': 'None',
    'maintainer_email': 'None',
    'url': 'None',
    'packages': packages,
    'package_data': package_data,
    'install_requires': install_requires,
    'entry_points': entry_points,
    'python_requires': '>=3.10,<4.0',
}


setup(**setup_kwargs)
