[
  {
    "objectID": "tutorials/python.html",
    "href": "tutorials/python.html",
    "title": "Tutorial: Python",
    "section": "",
    "text": "Install Ibis Birdbrain\n\n\n\n\nYou can use Ibis Birdbrain as a Python library, but it is often used through an interactive Python session. You can see how to start one with the birdbrain CLI with Ibis Birdbrain ready to use.\n\n\n\nExplore some data with Ibis Birdbrain and let us know how it goes!",
    "crumbs": [
      "Tutorial: Python"
    ]
  },
  {
    "objectID": "tutorials/python.html#prerequisites",
    "href": "tutorials/python.html#prerequisites",
    "title": "Tutorial: Python",
    "section": "",
    "text": "Install Ibis Birdbrain",
    "crumbs": [
      "Tutorial: Python"
    ]
  },
  {
    "objectID": "tutorials/python.html#overview",
    "href": "tutorials/python.html#overview",
    "title": "Tutorial: Python",
    "section": "",
    "text": "You can use Ibis Birdbrain as a Python library, but it is often used through an interactive Python session. You can see how to start one with the birdbrain CLI with Ibis Birdbrain ready to use.",
    "crumbs": [
      "Tutorial: Python"
    ]
  },
  {
    "objectID": "tutorials/python.html#getting-help",
    "href": "tutorials/python.html#getting-help",
    "title": "Tutorial: Python",
    "section": "",
    "text": "You can ask the bot for help with its usage:\nbot(\"how do I use you?\")\n\nbirdbrain:\nI’m birdbrain, your assistant for data analysis projects using Ibis, AI, and many other data tools. Here’s how you can interact with me:\n\nAsk Questions: You can ask me anything related to Ibis, AI, and data analysis. For example, you can ask about how to use certain features in Ibis, how to code in Python, or ask for help in debugging your code.\nRun Analysis: You can also ask me to run data analysis for you. You can give me instructions to query data, analyze it, and even generate visualizations.\nLearning Resources: If you’re looking for resources to learn more about Ibis, AI, and data analysis, I can provide you with relevant links to documentation, tutorials, and more.\nFile Operations: I can help with file operations such as reading a file, writing to a file, listing files in a directory, etc.\nTranslation & Summarization: If you have a large piece of text or a document that you want to summarize, or if you want to translate text from one language to another, I can help with that too.\n\nRemember, I can only provide accurate information and perform tasks based on the data I have access to. I’m here to assist you. Let’s explore the world of data together!",
    "crumbs": [
      "Tutorial: Python"
    ]
  },
  {
    "objectID": "tutorials/python.html#using-a-tool",
    "href": "tutorials/python.html#using-a-tool",
    "title": "Tutorial: Python",
    "section": "",
    "text": "Interact via natural language to have the bot use tools on your behalf.\n\n\nIbis Birdbrain can read data files (limited to Delta Lake tables and Excel files, currently), registering them as tables in your Ibis backend. If you connect to a data platform with existing tables via Ibis, you will have access to them through the bot.\nIn this example, we’ll read in a local penguins.xlsx file and ask the bot about it:\nbot(\"read in the penguins excel\")\n\nbirdbrain:\nI’ve successfully read in the penguins Excel file. Here’s a preview of the data:\n\n\n\n\n\n\n\n\n\n\n\n\n\nspecies\nisland\nbill_length_mm\nbill_depth_mm\nflipper_length_mm\nbody_mass_g\nsex\nyear\n\n\n\n\nAdelie\nTorgersen\n39.1\n18.7\n181\n3750\nmale\n2007\n\n\nAdelie\nTorgersen\n39.5\n17.4\n186\n3800\nfemale\n2007\n\n\nAdelie\nTorgersen\n40.3\n18.0\n195\n3250\nfemale\n2007\n\n\nAdelie\nTorgersen\nNaN\nNaN\n-\n-\n-\n2007\n\n\nAdelie\nTorgersen\n36.7\n19.3\n193\n3450\nfemale\n2007\n\n\n\n(and so on…) ) Note: Null values are represented as NaN (Not a Number) or ‘-’ for numerical columns, and as NULL for string columns.\n\nbot(\"what tables do you have access to? give an overview of the data\")\n\nbirdbrain:\nCurrently, I have access to one table named ‘penguins’.\nHere’s an overview of the data in it:\n\nspecies: This is a string column that gives the species of the penguin.\nisland: This is a string column that mentions the island the penguin is from.\nbill_length_mm: This is a float64 column that specifies the length of the penguin’s bill in millimeters. 4. bill_depth_mm: This is a float64 column that specifies the depth of the penguin’s bill in millimeters.\nflipper_length_mm: This is an int64 column that provides the length of the penguin’s flipper in millimeters.\nbody_mass_g: This is an int64 column that gives the body mass of the penguin in grams.\nsex: This is a string column that mentions the sex of the penguin.\nyear: This is an int64 column that gives the year of observation.\n\nPlease note that a string field can contain text, a float64 field can contain any real number (including decimal values), and an int64 field can contain any integer.\n\n\n\n\nbot(\"what is this dataset? search the internet and find out more about its origin for me\")\n\nbirdbrain:\nThe penguins dataset, a possible alternative to the classic Iris dataset, contains data on three species of penguins from three islands in the Palmer Archipelago, Antarctica [1][2][3].\nThe data comprises measurements such as bill length, bill depth, flipper length, body mass, along with categorical data such as species, island, and sex [3]. It provides a good mix of numeric and categorical data for various data exploration and machine learning tasks.\nThe dataset was originally collected and made available by Dr. Kristen Gorman and the Palmer Station, Antarctica, Long Term Ecological Research (LTER) [2].\nThe purpose of the Palmer LTER is to study the marine ecosystem in the western Antarctic Peninsula region as a microcosm of global ocean ecosystems, focusing on the processes that link the extent of annual sea ice to the biological dynamics of different trophic levels [2].\nSources: [1] Penguin dataset : The new Iris | Kaggle [2] (https://allisonhorst.github.io/palmerpenguins/) [3] Penguins Dataset Overview — iris alternative | by AbdulMajedRaja RS | Towards Data Science\n\n\n\n\nbot(\"read in the readme; I don't think it contians much, draft a better version for me based on this session\")\n\nbirdbrain:\nThe ‘penguins’ project now has a new and improved README:\n\nThis repository contains an exploration and analysis of the penguins dataset. The data was originally collected and made available by Dr. Kristen Gorman and the Palmer Station, Antarctica, Long Term Ecological Research (LTER).\nThe penguins dataset provides detailed data on three species of penguins from three islands in the Palmer Archipelago, Antarctica. It includes measurements such as bill length, bill depth, flipper length, body mass, along with categorical data such as species, island, and sex. The dataset presents a rich mix of numeric and categorical data, making it an excellent resource for various data exploration and machine learning tasks.\n\nThe data for this project is stored in an Excel file named ‘penguins.xlsx’ and is loaded into the project using Python. The data is then available for analysis and visualization.\n\nDetailed analysis on this data is yet to be performed. The objective is to derive meaningful insights about penguin species, their physical characteristics, and their distribution across the islands.\n\nContributions are welcome. Please read the CONTRIBUTING.md for details.\n\nThis project is licensed under the terms of the MIT License.\nThis summary gives a good overview of the data involved in the project, the purpose of the project, and how others can contribute.\n\nbot(\"only capitalize the first word in headings; remove the contributing/license stuff, this is buried deep in a repo. simplify and just include basic info, then write the file out for me\")\n\nbirdbrain:\nHere’s a simplified version of the README:\n\nThis project involves an exploration and analysis of the penguins dataset. The dataset, originally collected by Dr. Kristen Gorman and the Palmer Station, Antarctica, Long Term Ecological Research (LTER), provides detailed data on three species of penguins from three islands in the Palmer Archipelago, Antarctica.\n\nThe penguins dataset includes measurements such as bill length, bill depth, flipper length, body mass, and categorical data such as species, island, and sex. The data is stored in an Excel file named ‘penguins.xlsx’ and is loaded into the project using Python.\n\nThe project aims to use this data to derive meaningful insights about penguin species, their physical characteristics, and their distribution across the islands.\nI’ll now write this content to the README file.\n\nbot(\"write away!\")\n\nbirdbrain:\nI’ve successfully written the updated README to the file ./README.md.\n\ncat README.md\n\n\nThis project involves an exploration and analysis of the penguins dataset. The dataset, originally collected by Dr. Kristen Gorman and the Palmer Station, Antarctica, Long Term Ecological Research (LTER), provides detailed data on three species of penguins from three islands in the Palmer Archipelago, Antarctica.\n\nThe penguins dataset includes measurements such as bill length, bill depth, flipper length, body mass, and categorical data such as species, island, and sex. The data is stored in an Excel file named ‘penguins.xlsx’ and is loaded into the project using Python.\n\nThe project aims to use this data to derive meaningful insights about penguin species, their physical characteristics, and their distribution across the islands.",
    "crumbs": [
      "Tutorial: Python"
    ]
  },
  {
    "objectID": "tutorials/python.html#saving-and-loading-conversation-history",
    "href": "tutorials/python.html#saving-and-loading-conversation-history",
    "title": "Tutorial: Python",
    "section": "",
    "text": "From a new session:\n[ins] In [1]: bot(\"read in the readme\")\n\nbirdbrain:\nThe README file contains information about a data analysis project involving the penguins dataset. This dataset, collected by Dr. Kristen Gorman and the Palmer Station, Antarctica, Long Term Ecological Research (LTER), includes details on three species of penguins from three islands in the Palmer Archipelago, Antarctica.\nThe dataset contains measurements such as bill length, bill depth, flipper length, body mass, and categorical data such as species, island, and sex. The data is stored in an Excel file named ‘penguins.xlsx’ and is loaded into the project using Python.\nThe project aims to use this data to derive meaningful insights about the penguin species, their physical characteristics, and their distribution across the islands.\n\n[ins] In [2]: bot.save_history()\n\n[ins] In [3]: bb_con = ibis.connect(\"duckdb://birdbrain.ddb\")\n\n[ins] In [4]: bb_con.list_tables()\nOut[4]: ['history']\n\n[ins] In [5]: t = bb_con.table(\"history\")\n\n[ins] In [6]: t\nOut[6]:\n┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓\n┃ convo_id                   ┃ convo_timestamp            ┃ bot_name  ┃ message_content           ┃ message_timestamp                ┃ function_name       ┃\n┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩\n│ string                     │ timestamp                  │ string    │ string                    │ timestamp('UTC')                 │ string              │\n├────────────────────────────┼────────────────────────────┼───────────┼───────────────────────────┼──────────────────────────────────┼─────────────────────┤\n│ cc25b8fd-74b0-4184-bfde-7… │ 2023-10-01 09:40:47.736675 │ birdbrain │ read in the readme        │ 2023-10-01 16:40:31.961856+00:00 │ ~                   │\n│ cc25b8fd-74b0-4184-bfde-7… │ 2023-10-01 09:40:47.736675 │ birdbrain │ ['./requirements.txt', '… │ 2023-10-01 16:40:34.391470+00:00 │ list_files_and_dirs │\n│ cc25b8fd-74b0-4184-bfde-7… │ 2023-10-01 09:40:47.736675 │ birdbrain │ # Penguins data analysis… │ 2023-10-01 16:40:36.201490+00:00 │ read_file           │\n│ cc25b8fd-74b0-4184-bfde-7… │ 2023-10-01 09:40:47.736675 │ birdbrain │ The README file contains… │ 2023-10-01 16:40:43.166277+00:00 │ ~                   │\n└────────────────────────────┴────────────────────────────┴───────────┴───────────────────────────┴──────────────────────────────────┴─────────────────────┘",
    "crumbs": [
      "Tutorial: Python"
    ]
  },
  {
    "objectID": "tutorials/python.html#understanding-and-accessing-bot-objects",
    "href": "tutorials/python.html#understanding-and-accessing-bot-objects",
    "title": "Tutorial: Python",
    "section": "",
    "text": "You can ask the bot what tools it has access to and you should get an accurate answer, but you can also directly check. The Bot object is a very thin wrapper around a Marvin AI Application. You can access the Bot.ai attribute to get the underlying AIApplication object.\n[ins] In [1]: print(bot.ai.tools)\n[\n    Tool(\n        name='open_browser',\n        description='Opens the URL in a web browser.',\n        fn=&lt;function open_browser at 0x11f22fc40&gt;\n    ),\n    Tool(\n        name='search_internet',\n        description='Searches the internet for the given query.',\n        fn=&lt;function search_internet at 0x104890400&gt;\n    ),\n    Tool(\n        name='webpage_to_str',\n        description='Reads a webpage link into a string. Useful for summarizing\nwebpages.',\n        fn=&lt;function webpage_to_str at 0x11f22fba0&gt;\n    ),\n    Tool(\n        name='summarize_text',\n        description='Summarizes text',\n        fn=&lt;function summarize_text at 0x11f22fe20&gt;\n    ),\n    Tool(\n        name='translate_text',\n        description='Translates text',\n        fn=&lt;function translate_text at 0x11f22fec0&gt;\n    ),\n    Tool(\n        name='read_file',\n        description='Reads a file and returns its content.',\n        fn=&lt;function read_file at 0x11f25d260&gt;\n    ),\n    Tool(\n        name='list_files_and_dirs',\n        description='Lists all files and directories in a directory.',\n        fn=&lt;function list_files_and_dirs at 0x11f25d440&gt;\n    ),\n    Tool(\n        name='write_file',\n        description='Writes a file.',\n        fn=&lt;function write_file at 0x11f25d4e0&gt;\n    ),\n    Tool(\n        name='text_to_python',\n        description='Returns a Python code given text.',\n        fn=&lt;function text_to_python at 0x11f22e3e0&gt;\n    ),\n    Tool(\n        name='fix_python_error',\n        description='Fixes a Python error in the code.',\n        fn=&lt;function fix_python_error at 0x11f25cb80&gt;\n    ),\n    Tool(\n        name='run_python_code',\n        description='Execute Python code as a string and return the output',\n        fn=&lt;function run_python_code at 0x11f25ce00&gt;\n    ),\n    Tool(\n        name='list_tables',\n        description='Returns a list of available tables to query',\n        fn=&lt;function list_tables at 0x12a16e840&gt;\n    ),\n    Tool(\n        name='query_table',\n        description='Queries the table in the database to answer the question',\n        fn=&lt;function query_table at 0x12a16e980&gt;\n    ),\n    Tool(\n        name='get_table_schema',\n        description='Returns the schema of a table',\n        fn=&lt;function get_table_schema at 0x12a16e5c0&gt;\n    ),\n    Tool(\n        name='read_delta_table',\n        description='Reads a Delta Lake table directory from the full filepath\\n\\n\nfilepath should be of the form: &lt;path&gt;/&lt;table_name&gt;\\n    ',\n        fn=&lt;function read_delta_table at 0x12a16e7a0&gt;\n    ),\n    Tool(\n        name='read_excel_file',\n        description='Reads an Excel file from the full filepath\\n\\n    filepath\nshould be of the form: &lt;path&gt;/&lt;table_name&gt;.&lt;extension&gt;\\n    ',\n        fn=&lt;function read_excel_file at 0x12a16e8e0&gt;\n    ),\n    Tool(\n        name='read_files_and_summarize',\n        description='Read a list of files and summarize them.',\n        fn=&lt;function read_files_and_summarize at 0x12a16eb60&gt;\n    ),\n    Tool(\n        name='read_webpage_and_summarize',\n        description='Read a webpage and summarize it.',\n        fn=&lt;function read_webpage_and_summarize at 0x12a16ea20&gt;\n    )\n]",
    "crumbs": [
      "Tutorial: Python"
    ]
  },
  {
    "objectID": "tutorials/python.html#next-steps",
    "href": "tutorials/python.html#next-steps",
    "title": "Tutorial: Python",
    "section": "",
    "text": "Explore some data with Ibis Birdbrain and let us know how it goes!",
    "crumbs": [
      "Tutorial: Python"
    ]
  },
  {
    "objectID": "posts.html",
    "href": "posts.html",
    "title": "Posts",
    "section": "",
    "text": "Order By\n       Default\n         \n          Title\n        \n         \n          Date - Oldest\n        \n         \n          Date - Newest\n        \n         \n          Author\n        \n     \n  \n    \n      \n      \n    \n\n\n\n\n\n\n\n\nAuxiliary tools\n\n\n\nLLMs and data\n\n\n\n\n\n\n\nCody Peterson\n\n\nOct 16, 2023\n\n\n\n\n\n\n\n\n\n\n\n\nComposable data + AI systems\n\n\n\nLLMs and data\n\n\n\n\n\n\n\nCody Peterson\n\n\nOct 15, 2023\n\n\n\n\n\n\n\n\n\n\n\n\nComputations and control flow: it’s just programming\n\n\n\nLLMs and data\n\n\n\n\n\n\n\nCody Peterson\n\n\nOct 14, 2023\n\n\n\n\n\n\n\n\n\n\n\n\nThree approaches\n\n\n\nLLMs and data\n\n\n\n\n\n\n\nCody Peterson\n\n\nOct 13, 2023\n\n\n\n\n\n\n\n\n\n\n\n\nAn introduction to Marvin and Ibis\n\n\n\nLLMs and data\n\n\n\n\n\n\n\nCody Peterson\n\n\nOct 12, 2023\n\n\n\n\n\n\n\n\nNo matching items\n\n Back to top"
  },
  {
    "objectID": "concepts/ops.html",
    "href": "concepts/ops.html",
    "title": "LLMOps, MLOps, DevOps",
    "section": "",
    "text": "The ML landscape is plagued by “the toy problem” – building something cool is easy, putting something in production is hard.",
    "crumbs": [
      "LLMOps, MLOps, DevOps"
    ]
  },
  {
    "objectID": "concepts/platforms.html",
    "href": "concepts/platforms.html",
    "title": "Data and AI platforms",
    "section": "",
    "text": "Data and AI platforms are…",
    "crumbs": [
      "Data and AI platforms"
    ]
  },
  {
    "objectID": "contribute/contributing.html",
    "href": "contribute/contributing.html",
    "title": "Contributing",
    "section": "",
    "text": "To contribute…",
    "crumbs": [
      "Contribute",
      "Contributing"
    ]
  },
  {
    "objectID": "contribute/contributing.html#required-dependencies",
    "href": "contribute/contributing.html#required-dependencies",
    "title": "Contributing",
    "section": "",
    "text": "To contribute…",
    "crumbs": [
      "Contribute",
      "Contributing"
    ]
  },
  {
    "objectID": "install.html",
    "href": "install.html",
    "title": "Installation and setup",
    "section": "",
    "text": "This page describes how to install and setup Ibis Birdbrain.\n\n\npip install ibis-birdbrain\n\n\n\n\n\n\n\n\n\nWarning\n\n\n\nDocumentation coming soon!\n\n\n\n\n\n\n\n\nWarning\n\n\n\nOnly DuckDB supported for now. All 18+ Ibis backends coming soon.\n\n\n\n\n\n\n\n\n\n\n\nWarning\n\n\n\nDocumentation coming soon!\n\n\n\n\n\n\n\n\nWarning\n\n\n\nOnly Azure OpenAI is supported for now. All Marvin providers coming soon.\n\n\nCreate the ~/.birdbrain/.env file:\nMARVIN_AZURE_OPENAI_API_KEY=\"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\"\nMARVIN_AZURE_OPENAI_API_BASE=\"https://birdbrain.openai.azure.com\"\nMARVIN_AZURE_OPENAI_DEPLOYMENT_NAME=\"birdbrain-4-32k\"\nMARVIN_AZURE_OPENAI_API_TYPE=\"azure\"\n\n\n\nLearn how get started with Ibis Birdbrain’s CLI.",
    "crumbs": [
      "Installation and setup"
    ]
  },
  {
    "objectID": "install.html#install-from-pypi",
    "href": "install.html#install-from-pypi",
    "title": "Installation and setup",
    "section": "",
    "text": "pip install ibis-birdbrain",
    "crumbs": [
      "Installation and setup"
    ]
  },
  {
    "objectID": "install.html#data-platform-setup",
    "href": "install.html#data-platform-setup",
    "title": "Installation and setup",
    "section": "",
    "text": "Warning\n\n\n\nDocumentation coming soon!\n\n\n\n\n\n\n\n\nWarning\n\n\n\nOnly DuckDB supported for now. All 18+ Ibis backends coming soon.",
    "crumbs": [
      "Installation and setup"
    ]
  },
  {
    "objectID": "install.html#ai-platform-setup",
    "href": "install.html#ai-platform-setup",
    "title": "Installation and setup",
    "section": "",
    "text": "Warning\n\n\n\nDocumentation coming soon!\n\n\n\n\n\n\n\n\nWarning\n\n\n\nOnly Azure OpenAI is supported for now. All Marvin providers coming soon.\n\n\nCreate the ~/.birdbrain/.env file:\nMARVIN_AZURE_OPENAI_API_KEY=\"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\"\nMARVIN_AZURE_OPENAI_API_BASE=\"https://birdbrain.openai.azure.com\"\nMARVIN_AZURE_OPENAI_DEPLOYMENT_NAME=\"birdbrain-4-32k\"\nMARVIN_AZURE_OPENAI_API_TYPE=\"azure\"",
    "crumbs": [
      "Installation and setup"
    ]
  },
  {
    "objectID": "install.html#next-steps",
    "href": "install.html#next-steps",
    "title": "Installation and setup",
    "section": "",
    "text": "Learn how get started with Ibis Birdbrain’s CLI.",
    "crumbs": [
      "Installation and setup"
    ]
  },
  {
    "objectID": "why.html",
    "href": "why.html",
    "title": "Why Ibis Birdbrain?",
    "section": "",
    "text": "Ibis is the portable Python dataframe library.\nIbis Birdbrain is the portable Python LM-powered data bot, built on Ibis with support for all data platforms Ibis’s 20+ backends support. Ibis Birdbrain is also built on Marvin with support for all AI platforms Marvin supports.\n\n\nLangauge models have…\n\n\n\n\n\nWhen things don’t work as they should, it often means that standards are absent.\n- The International Organization for Standardization (ISO)\n\n- The Composable Codex\n\n\n\n\nComposable data systems are MICE: modular, interoperable, customizable, and extensible.\nIbis Birdbrain aims to bring these values to a data + AI framework that makes working with LLMs delightful and easy, for individual hobbiests or the more advanced technology organizations.",
    "crumbs": [
      "Why Ibis Birdbrain?"
    ]
  },
  {
    "objectID": "why.html#standards",
    "href": "why.html#standards",
    "title": "Why Ibis Birdbrain?",
    "section": "",
    "text": "When things don’t work as they should, it often means that standards are absent.\n- The International Organization for Standardization (ISO)\n\n- The Composable Codex",
    "crumbs": [
      "Why Ibis Birdbrain?"
    ]
  },
  {
    "objectID": "why.html#composable-data-systems-are-mice",
    "href": "why.html#composable-data-systems-are-mice",
    "title": "Why Ibis Birdbrain?",
    "section": "",
    "text": "Composable data systems are MICE: modular, interoperable, customizable, and extensible.\nIbis Birdbrain aims to bring these values to a data + AI framework that makes working with LLMs delightful and easy, for individual hobbiests or the more advanced technology organizations.",
    "crumbs": [
      "Why Ibis Birdbrain?"
    ]
  },
  {
    "objectID": "posts/llms-and-data-pt2/index.html",
    "href": "posts/llms-and-data-pt2/index.html",
    "title": "Computations and control flow: it’s just programming",
    "section": "",
    "text": "The recent Generative AI hype cycle has led to a lot of new terminology to understand. In this post, we’ll cover some key concepts from the groud up and explain the basics of working with LLMs in the context of data.\nThis post assumes basic familiarity with Marvin and Ibis and three approaches to applying LLMs to data.\n\n\nCode\n1import ibis\nimport marvin\n\nfrom dotenv import load_dotenv\n\n2load_dotenv()\n\n3con = ibis.connect(\"duckdb://penguins.ddb\")\nt = ibis.examples.penguins.fetch()\nt = con.create_table(\"penguins\", t.to_pyarrow(), overwrite=True)\n\n\n\n1\n\nImport the libraries we need.\n\n2\n\nLoad the environment variable to setup Marvin to call our OpenAI account.\n\n3\n\nSetup the demo datain an Ibis backend.\n\n\n\n\nFirst, we’ll setup Ibis and Marvin with some simple example data:\n\n1import ibis\nimport marvin\n\nfrom ibis.expr.schema import Schema\nfrom ibis.expr.types.relations import Table\n\n2ibis.options.interactive = True\nmarvin.settings.llm_model = \"openai/gpt-4\"\n\n3con = ibis.connect(\"duckdb://penguins.ddb\")\nt = con.table(\"penguins\")\n\n\n1\n\nImport Ibis and Marvin.\n\n2\n\nConfigure Ibis (interactive) and Marvin (GPT-4).\n\n3\n\nConnect to the data and load a table into a variable."
  },
  {
    "objectID": "posts/llms-and-data-pt2/index.html#introduction",
    "href": "posts/llms-and-data-pt2/index.html#introduction",
    "title": "Computations and control flow: it’s just programming",
    "section": "",
    "text": "The recent Generative AI hype cycle has led to a lot of new terminology to understand. In this post, we’ll cover some key concepts from the groud up and explain the basics of working with LLMs in the context of data.\nThis post assumes basic familiarity with Marvin and Ibis and three approaches to applying LLMs to data.\n\n\nCode\n1import ibis\nimport marvin\n\nfrom dotenv import load_dotenv\n\n2load_dotenv()\n\n3con = ibis.connect(\"duckdb://penguins.ddb\")\nt = ibis.examples.penguins.fetch()\nt = con.create_table(\"penguins\", t.to_pyarrow(), overwrite=True)\n\n\n\n1\n\nImport the libraries we need.\n\n2\n\nLoad the environment variable to setup Marvin to call our OpenAI account.\n\n3\n\nSetup the demo datain an Ibis backend.\n\n\n\n\nFirst, we’ll setup Ibis and Marvin with some simple example data:\n\n1import ibis\nimport marvin\n\nfrom ibis.expr.schema import Schema\nfrom ibis.expr.types.relations import Table\n\n2ibis.options.interactive = True\nmarvin.settings.llm_model = \"openai/gpt-4\"\n\n3con = ibis.connect(\"duckdb://penguins.ddb\")\nt = con.table(\"penguins\")\n\n\n1\n\nImport Ibis and Marvin.\n\n2\n\nConfigure Ibis (interactive) and Marvin (GPT-4).\n\n3\n\nConnect to the data and load a table into a variable."
  },
  {
    "objectID": "posts/llms-and-data-pt2/index.html#context",
    "href": "posts/llms-and-data-pt2/index.html#context",
    "title": "Computations and control flow: it’s just programming",
    "section": "Context",
    "text": "Context\nContext is a fancy way of talking about the input to a LLM."
  },
  {
    "objectID": "posts/llms-and-data-pt2/index.html#calls",
    "href": "posts/llms-and-data-pt2/index.html#calls",
    "title": "Computations and control flow: it’s just programming",
    "section": "Calls",
    "text": "Calls\nWe make calls with inputs to functions or systems and get outputs. We can think of calling the LLM with our input (context) and getting an output (text)."
  },
  {
    "objectID": "posts/llms-and-data-pt2/index.html#computations",
    "href": "posts/llms-and-data-pt2/index.html#computations",
    "title": "Computations and control flow: it’s just programming",
    "section": "Computations",
    "text": "Computations\nA function or system often computes something. We can be pedantic about calls versus computations, but in general the connotation around computations is more time and resource intensive than a call. At the end of the day, they will both take some computer cycles."
  },
  {
    "objectID": "posts/llms-and-data-pt2/index.html#retrieval-augmented-generation-rag",
    "href": "posts/llms-and-data-pt2/index.html#retrieval-augmented-generation-rag",
    "title": "Computations and control flow: it’s just programming",
    "section": "Retrieval augmented generation (RAG)",
    "text": "Retrieval augmented generation (RAG)\nInstead of you typing out context for the bot, we can retrieve context from somewhere, augment our strings sent to the bot with this context, and then generate a response from the bot.\nAs a contrived example, instead of saying “The capitol of foo is bar”, we can retrieve the capitol of foo from a database, augment it with our context, and then generate a response from the bot. You may notice that we already did this in the firt post in the series – let’s review that code again:\n\nfrom ibis.expr.schema import Schema\nfrom ibis.expr.types.relations import Table\n\n\n@marvin.ai_fn\ndef sql_select(\n    text: str, table_name: str = t.get_name(), schema: Schema = t.schema()\n) -&gt; str:\n    \"\"\"writes the SQL SELECT statement to query the table according to the text\"\"\"\n\n\nquery = \"the unique combination of species and islands\"\nsql = sql_select(query).strip(\";\")\nsql\n\n'SELECT DISTINCT species, island FROM penguins'\n\n\nNotice that we retrieved the table name and schema with calls to the Ibis table (t.get_name() and t.schema()). We then augment our context (the query in natural language) with this information and generate a response from the bot.\nThis works reasonably well for simple SQL queries:\n\nt.sql(sql)\n\n┏━━━━━━━━━━━┳━━━━━━━━━━━┓\n┃ species   ┃ island    ┃\n┡━━━━━━━━━━━╇━━━━━━━━━━━┩\n│ string    │ string    │\n├───────────┼───────────┤\n│ Adelie    │ Torgersen │\n│ Adelie    │ Biscoe    │\n│ Adelie    │ Dream     │\n│ Gentoo    │ Biscoe    │\n│ Chinstrap │ Dream     │\n└───────────┴───────────┘\n\n\n\nI would argue in this case there wasn’t any real computation done by our calls to the Ibis table – we were just retrieving some relatively static metadata – but we could have done some more complex computations (on any of 18+ data platforms)."
  },
  {
    "objectID": "posts/llms-and-data-pt2/index.html#thought-leadership",
    "href": "posts/llms-and-data-pt2/index.html#thought-leadership",
    "title": "Computations and control flow: it’s just programming",
    "section": "Thought leadership",
    "text": "Thought leadership\nTODO: human rewrite\nIn the realm of Generative AI, particularly when working with Language Learning Models (LLMs), understanding the concept of ‘context’ is crucial. Context, in this domain, refers to the inputs that are fed into an LLM, and the corresponding outputs they generate. This post breaks down the complexities of this process into understandable fragments, including retrieval of context, its augmentation, and, thereafter, the generation of a response.\nAn illustrative example is provided, showcasing a database interaction. It demonstrates how the data retrieved can be used to augment the context before the bot generates a response. This valuable insight underlines the practical application of the theory, reinforcing the understanding of the readers.\nWe also venture into the difference between simple static metadata retrieval and the more intricate computations. This distinction echoes the breadth and depth of the processes involved in Generative AI.\nAs we continue to explore and unravel the potential of Generative AI and LLMs, this post serves as a fundamental building block. It creates a pathway for enthusiasts and professionals alike to delve deeper into this exciting field. By breaking down complex concepts into comprehensible segments, it fosters an environment of learning and growth.\nThis marks just the beginning of our journey into the world of Generative AI. As we dig deeper, we will continue to explore, learn and share with our readers. Stay tuned for more insightful content in this series. [1]\n[1] https://github.com/ibis-project/ibis-birdbrain"
  },
  {
    "objectID": "posts/llms-and-data-pt2/index.html#next-steps",
    "href": "posts/llms-and-data-pt2/index.html#next-steps",
    "title": "Computations and control flow: it’s just programming",
    "section": "Next steps",
    "text": "Next steps\nYou can get involved with Ibis Birdbrain, our open-source data & AI project for building next-generation natural language interfaces to data.\nRead the next post in this series."
  },
  {
    "objectID": "posts/llms-and-data-pt4/index.html",
    "href": "posts/llms-and-data-pt4/index.html",
    "title": "Auxiliary tools",
    "section": "",
    "text": "As a product manager, I don’t spend most of my time managing products. I suspect most data developers (analysts, engineers, scientists, etc.) don’t spend most of their time writing data code. There are many auxiliary tasks that are required in doing many technical jobs. These include:\n\nsearching the Internet for information\nreading, summarizing, and synthesizing information\nperforming boring computer tasks\ntranslating between different languages (e.g. SQL and Python; English and Spanish)\ncopying and modifying existing code\nquerying some basic informatin from data platforms\n\nWhat if we could, through natural language, have a bot perform many of these tasks (in addition to basic data analysis) on our behalf?"
  },
  {
    "objectID": "posts/llms-and-data-pt4/index.html#introduction",
    "href": "posts/llms-and-data-pt4/index.html#introduction",
    "title": "Auxiliary tools",
    "section": "",
    "text": "As a product manager, I don’t spend most of my time managing products. I suspect most data developers (analysts, engineers, scientists, etc.) don’t spend most of their time writing data code. There are many auxiliary tasks that are required in doing many technical jobs. These include:\n\nsearching the Internet for information\nreading, summarizing, and synthesizing information\nperforming boring computer tasks\ntranslating between different languages (e.g. SQL and Python; English and Spanish)\ncopying and modifying existing code\nquerying some basic informatin from data platforms\n\nWhat if we could, through natural language, have a bot perform many of these tasks (in addition to basic data analysis) on our behalf?"
  },
  {
    "objectID": "posts/llms-and-data-pt4/index.html#were-using-python-lets-use-python",
    "href": "posts/llms-and-data-pt4/index.html#were-using-python-lets-use-python",
    "title": "Auxiliary tools",
    "section": "We’re using Python, let’s use Python",
    "text": "We’re using Python, let’s use Python\nWe’re already using Python for Ibis and Marvin. Let’s use it for auxillary tools. We’ll setup our data and AI platform connections and some simple example data to work with.\n\n\nCode\n1import ibis\nimport marvin\n\nfrom dotenv import load_dotenv\n\n2load_dotenv()\n\n3con = ibis.connect(\"duckdb://penguins.ddb\")\nt = ibis.examples.penguins.fetch()\nt = con.create_table(\"penguins\", t.to_pyarrow(), overwrite=True)\n\n\n\n1\n\nImport the libraries we need.\n\n2\n\nLoad the environment variable to setup Marvin to call our OpenAI account.\n\n3\n\nSetup the demo datain an Ibis backend.\n\n\n\n\n\n1import ibis\nimport marvin\n\nfrom ibis.expr.schema import Schema\nfrom ibis.expr.types.relations import Table\n\n2ibis.options.interactive = True\nmarvin.settings.llm_model = \"openai/gpt-4\"\n\n3con = ibis.connect(\"duckdb://penguins.ddb\")\nt = con.table(\"penguins\")\n\n\n1\n\nImport Ibis and Marvin.\n\n2\n\nConfigure Ibis (interactive) and Marvin (GPT-4).\n\n3\n\nConnect to the data and load a table into a variable."
  },
  {
    "objectID": "posts/llms-and-data-pt4/index.html#filesystem-tools",
    "href": "posts/llms-and-data-pt4/index.html#filesystem-tools",
    "title": "Auxiliary tools",
    "section": "Filesystem tools",
    "text": "Filesystem tools"
  },
  {
    "objectID": "posts/llms-and-data-pt4/index.html#internet-tools",
    "href": "posts/llms-and-data-pt4/index.html#internet-tools",
    "title": "Auxiliary tools",
    "section": "Internet tools",
    "text": "Internet tools"
  },
  {
    "objectID": "posts/llms-and-data-pt4/index.html#ai-powered-tools",
    "href": "posts/llms-and-data-pt4/index.html#ai-powered-tools",
    "title": "Auxiliary tools",
    "section": "AI-powered tools",
    "text": "AI-powered tools"
  },
  {
    "objectID": "posts/llms-and-data-pt4/index.html#introducing-ibis-birdbrain",
    "href": "posts/llms-and-data-pt4/index.html#introducing-ibis-birdbrain",
    "title": "Auxiliary tools",
    "section": "Introducing Ibis Birdbrain",
    "text": "Introducing Ibis Birdbrain\nIntroduce the bot. Need to overview the tools here I think, but should probably skip most details."
  },
  {
    "objectID": "posts/llms-and-data-pt4/index.html#a-comparison-with-mlops",
    "href": "posts/llms-and-data-pt4/index.html#a-comparison-with-mlops",
    "title": "Auxiliary tools",
    "section": "A comparison with MLOps",
    "text": "A comparison with MLOps\nTODO: point on how most of the work is not ML\nBefore “MLOps” was a standard term, the Sculley et al paper from 2015 described the key issues with building real-world ML systems.\n\n\n\nFigure 1: Only a small fraction of real-world ML systems is composed of the ML code, as shown by the small black box in the middle. The required surround infrastructure is vast and complex.\n\n\nYou can substitute “LLM” for “ML” in the above figure.\n\nThe “toy problem” problem\nML and LLMs are cool! They’re fun to play with and it’s easy to get distracted with fun applications. Often, ML is learned through solving toy problems, and …\n\n\nThe application landscape is vast\n…and thus requires modular, interoperable, customizable, and extensible tools. TODO: more comparison to MLOps."
  },
  {
    "objectID": "posts/llms-and-data-pt4/index.html#next-steps",
    "href": "posts/llms-and-data-pt4/index.html#next-steps",
    "title": "Auxiliary tools",
    "section": "Next steps",
    "text": "Next steps\nYou can get involved with Ibis Birdbrain, our open-source data & AI project for building next-generation natural language interfaces to data."
  },
  {
    "objectID": "posts/llms-and-data-pt0/index.html",
    "href": "posts/llms-and-data-pt0/index.html",
    "title": "An introduction to Marvin and Ibis",
    "section": "",
    "text": "In this “LLMs and data” series, we’ll explore how to apply large-language models (LLMs) to data analytics. We’ll walk through the steps to build Ibis Birdbrain.\nThroughout the series, we’ll be using Marvin and Ibis. A brief introduction to each is provided below."
  },
  {
    "objectID": "posts/llms-and-data-pt0/index.html#introduction",
    "href": "posts/llms-and-data-pt0/index.html#introduction",
    "title": "An introduction to Marvin and Ibis",
    "section": "",
    "text": "In this “LLMs and data” series, we’ll explore how to apply large-language models (LLMs) to data analytics. We’ll walk through the steps to build Ibis Birdbrain.\nThroughout the series, we’ll be using Marvin and Ibis. A brief introduction to each is provided below."
  },
  {
    "objectID": "posts/llms-and-data-pt0/index.html#marvin",
    "href": "posts/llms-and-data-pt0/index.html#marvin",
    "title": "An introduction to Marvin and Ibis",
    "section": "Marvin",
    "text": "Marvin\nMarvin is an AI engineering framework that makes it easy to build up to an interactive conversational application.\nMarvin makes calls to an AI platform. You typically use an API key set as an environment variable – in this case, we’ll load a .env file that contians secrets for the AI platform that Marvin will use. We also set the large language model model.\n\n1import marvin\n\nfrom rich import print\nfrom time import sleep\nfrom dotenv import load_dotenv\n\n2load_dotenv()\n\n# increase accuracy\n3marvin.settings.llm_model = \"openai/gpt-4\"\n# decrease cost\n# marvin.settings.llm_model = \"openai/gpt-3.5-turbo\"\n\n4test_str = \"working with data and LLMs on 18+ data platforms is easy!\"\ntest_str\n\n\n1\n\nImport the libraries we need.\n\n2\n\nLoad the environment variable to setup Marvin to call our OpenAI account.\n\n3\n\nConfigure the LLM model to use.\n\n4\n\nSome text to test on\n\n\n\n\n'working with data and LLMs on 18+ data platforms is easy!'\n\n\n\nFunctions\nAI functions are one of the building blocks in Marvin and allow yout to specify a typed python function with no code – only a docstring – to achieve a wide variety of tasks.\nWe’ll demonstrate this with an AI function that trnaslates text:\n\n@marvin.ai_fn\ndef translate(text: str, from_: str = \"English\", to: str = \"Spanish\") -&gt; str:\n    \"\"\"translates the text\"\"\"\n\ntranslate(test_str)\n\n'trabajar con datos y LLMs en más de 18 plataformas de datos es fácil!'\n\n\n\n\nCode\n1sleep(1)\n\n\n\n1\n\nAvoid rate-limiting by waiting.\n\n\n\n\n\ntranslate(translate(test_str), from_=\"Spanish\", to=\"English\")\n\n'Working with data and LLMs on more than 18 data platforms is easy!'\n\n\n\n\nCode\n1sleep(3)\n\n\n\n1\n\nAvoid rate-limiting by waiting.\n\n\n\n\n\n\nModels\nAI models are another building block for generating python classes from input text. It’s a great way to build structured data from unstructured data that can be customized for your needs.\nWe’ll demosntrate this with an AI model that translates text:\n\nfrom pydantic import BaseModel, Field\n\n# decrease cost\nmarvin.settings.llm_model = \"openai/gpt-3.5-turbo\"\n\n@marvin.ai_model\nclass ExtractParts(BaseModel):\n    \"\"\"Extracts parts of a sentence\"\"\"\n    subject: str = Field(..., description=\"The subject of the sentence.\")\n    objects: list[str] = Field(..., description=\"The objects of the sentence.\")\n    predicate: str = Field(..., description=\"The predicate of the sentence.\")\n    modifiers: list[str] = Field(..., description=\"The modifiers of the sentence.\")\n\nExtractParts(test_str)\n\nExtractParts(subject='working', objects=['data', 'LLMs'], predicate='is', modifiers=['on 18+ data platforms', 'easy'])\n\n\n\n\nCode\n1sleep(1)\n\n\n\n1\n\nAvoid rate-limiting by waiting.\n\n\n\n\n\n\nClassifiers\nAI classifiers are another building block for generating python classes from input text. It’s the most efficient (time and cost) method for applying LLMs as it only results in a single output token, selecting an output in a specified Enum.\nWe’ll demonstrate this by classifying the language of some text:\n\nfrom enum import Enum\n\n# increase accuracy\nmarvin.settings.llm_model = \"openai/gpt-4\"\n\n@marvin.ai_classifier\nclass IdentifyLanguage(Enum):\n    \"\"\"Identifies the language of the text\"\"\"\n\n    english = \"English\"\n    spanish = \"Spanish\"\n\n\nIdentifyLanguage(test_str).value\n\n'English'\n\n\n\n\nCode\n1sleep(1)\n\n\n\n1\n\nAvoid rate-limiting by waiting.\n\n\n\n\n\nIdentifyLanguage(translate(test_str)).value\n\n'Spanish'\n\n\n\n\nCode\n1sleep(3)\n\n\n\n1\n\nAvoid rate-limiting by waiting."
  },
  {
    "objectID": "posts/llms-and-data-pt0/index.html#ibis",
    "href": "posts/llms-and-data-pt0/index.html#ibis",
    "title": "An introduction to Marvin and Ibis",
    "section": "Ibis",
    "text": "Ibis\nIbis is the portable Python dataframe library that enables Ibis Birdbrain to work on many data platforms at native scale.\nIbis makes calls to a data platform, providing an API but pushing the compute to (local or remote) query engines and storage. DuckDB is the default and we’ll typically use it for demo puroses. You can work with an in-memory instance, but we’ll often create a database file from example data:\n\n1import ibis\n\n2con = ibis.connect(\"duckdb://penguins.ddb\")\nt = ibis.examples.penguins.fetch()\nt = con.create_table(\"penguins\", t.to_pyarrow(), overwrite=True)\n\n\n1\n\nImport the libraries we need.\n\n2\n\nSetup the demo datain an Ibis backend.\n\n\n\n\nYou will typically connect to an existing data platform via your corresponding Ibis backend and have access to a number of tables:\n\n1import ibis\n\n2ibis.options.interactive = True\n\n3con = ibis.connect(\"duckdb://penguins.ddb\")\nt = con.table(\"penguins\")\n\n\n1\n\nImport Ibis.\n\n2\n\nConfigure Ibis (interactive).\n\n3\n\nConnect to the data and load a table into a variable.\n\n\n\n\n\nBackend\nA backend provides the connection and basic management of the data platform. Above, we created the con variable that is an instance of a DuckDB backend:\n\ncon\n\n&lt;ibis.backends.duckdb.Backend at 0x16a17af10&gt;\n\n\nIt usually contains some tables:\n\ncon.list_tables()\n\n['penguins']\n\n\nWe can access some internals of Ibis to see what backends are available:\n\n\n\n\n\n\nTip\n\n\n\nDon’t rely on accessing internals of Ibis in production.\n\n\n\nbackends = [entrypoint.name for entrypoint in ibis.util.backend_entry_points()]\nbackends\n\n['bigquery',\n 'clickhouse',\n 'dask',\n 'datafusion',\n 'druid',\n 'duckdb',\n 'flink',\n 'impala',\n 'mssql',\n 'mysql',\n 'oracle',\n 'pandas',\n 'polars',\n 'postgres',\n 'pyspark',\n 'snowflake',\n 'sqlite',\n 'trino']\n\n\n\n\nTable\nYou typically work with a table, conventionally named t for demo or exploratory purposes:\n\nt\n\n┏━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┓\n┃ species ┃ island    ┃ bill_length_mm ┃ bill_depth_mm ┃ flipper_length_mm ┃ body_mass_g ┃ sex    ┃ year  ┃\n┡━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━┩\n│ string  │ string    │ float64        │ float64       │ int64             │ int64       │ string │ int64 │\n├─────────┼───────────┼────────────────┼───────────────┼───────────────────┼─────────────┼────────┼───────┤\n│ Adelie  │ Torgersen │           39.1 │          18.7 │               181 │        3750 │ male   │  2007 │\n│ Adelie  │ Torgersen │           39.5 │          17.4 │               186 │        3800 │ female │  2007 │\n│ Adelie  │ Torgersen │           40.3 │          18.0 │               195 │        3250 │ female │  2007 │\n│ Adelie  │ Torgersen │            nan │           nan │              NULL │        NULL │ NULL   │  2007 │\n│ Adelie  │ Torgersen │           36.7 │          19.3 │               193 │        3450 │ female │  2007 │\n│ Adelie  │ Torgersen │           39.3 │          20.6 │               190 │        3650 │ male   │  2007 │\n│ Adelie  │ Torgersen │           38.9 │          17.8 │               181 │        3625 │ female │  2007 │\n│ Adelie  │ Torgersen │           39.2 │          19.6 │               195 │        4675 │ male   │  2007 │\n│ Adelie  │ Torgersen │           34.1 │          18.1 │               193 │        3475 │ NULL   │  2007 │\n│ Adelie  │ Torgersen │           42.0 │          20.2 │               190 │        4250 │ NULL   │  2007 │\n│ …       │ …         │              … │             … │                 … │           … │ …      │     … │\n└─────────┴───────────┴────────────────┴───────────────┴───────────────────┴─────────────┴────────┴───────┘\n\n\n\nWhen working with many tables, you should name them descriptively.\n\n\nSchema\nA table has a schema that Ibis maps to the data platform’s data types:\n\nt.schema()\n\nibis.Schema {\n  species            string\n  island             string\n  bill_length_mm     float64\n  bill_depth_mm      float64\n  flipper_length_mm  int64\n  body_mass_g        int64\n  sex                string\n  year               int64\n}"
  },
  {
    "objectID": "posts/llms-and-data-pt0/index.html#llms-and-data-marvin-and-ibis",
    "href": "posts/llms-and-data-pt0/index.html#llms-and-data-marvin-and-ibis",
    "title": "An introduction to Marvin and Ibis",
    "section": "LLMs and data: Marvin and Ibis",
    "text": "LLMs and data: Marvin and Ibis\nYou can use Marvin and Ibis together to easily apply LLMs to data.\n\nfrom ibis.expr.schema import Schema\nfrom ibis.expr.types.relations import Table\n\n@marvin.ai_fn\ndef sql_select(\n    text: str, table_name: str = t.get_name(), schema: Schema = t.schema()\n) -&gt; str:\n    \"\"\"writes the SQL SELECT statement to query the table according to the text\"\"\"\n\n\nquery = \"the unique combination of species and islands\"\nsql = sql_select(query).strip(\";\")\nsql\n\n'SELECT DISTINCT species, island FROM penguins'\n\n\n\nt.sql(sql)\n\n┏━━━━━━━━━━━┳━━━━━━━━━━━┓\n┃ species   ┃ island    ┃\n┡━━━━━━━━━━━╇━━━━━━━━━━━┩\n│ string    │ string    │\n├───────────┼───────────┤\n│ Adelie    │ Torgersen │\n│ Adelie    │ Biscoe    │\n│ Adelie    │ Dream     │\n│ Gentoo    │ Biscoe    │\n│ Chinstrap │ Dream     │\n└───────────┴───────────┘\n\n\n\n\n\nCode\n1sleep(3)\n\n\n\n1\n\nAvoid rate-limiting by waiting.\n\n\n\n\n\nt.sql(sql_select(query + \" and include their counts in from highest to lowest\").strip(\";\"))\n\n┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━┓\n┃ species   ┃ island    ┃ count ┃\n┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━┩\n│ string    │ string    │ int64 │\n├───────────┼───────────┼───────┤\n│ Gentoo    │ Biscoe    │   124 │\n│ Chinstrap │ Dream     │    68 │\n│ Adelie    │ Dream     │    56 │\n│ Adelie    │ Torgersen │    52 │\n│ Adelie    │ Biscoe    │    44 │\n└───────────┴───────────┴───────┘"
  },
  {
    "objectID": "posts/llms-and-data-pt0/index.html#next-steps",
    "href": "posts/llms-and-data-pt0/index.html#next-steps",
    "title": "An introduction to Marvin and Ibis",
    "section": "Next steps",
    "text": "Next steps\nYou can get involved with Ibis Birdbrain, our open-source data & AI project for building next-generation natural language interfaces to data.\nRead the next post in this series."
  },
  {
    "objectID": "posts/llms-and-data-pt1/index.html",
    "href": "posts/llms-and-data-pt1/index.html",
    "title": "Three approaches",
    "section": "",
    "text": "The thought of using natural language to transform and analyze data is appealing. This post assumes familiarity with Marvin and Ibis – read the previous post in the series for a quick overview."
  },
  {
    "objectID": "posts/llms-and-data-pt1/index.html#introduction",
    "href": "posts/llms-and-data-pt1/index.html#introduction",
    "title": "Three approaches",
    "section": "",
    "text": "The thought of using natural language to transform and analyze data is appealing. This post assumes familiarity with Marvin and Ibis – read the previous post in the series for a quick overview."
  },
  {
    "objectID": "posts/llms-and-data-pt1/index.html#approaches",
    "href": "posts/llms-and-data-pt1/index.html#approaches",
    "title": "Three approaches",
    "section": "Approaches",
    "text": "Approaches\nWhen discussed at Voltron Data, we identified three distinct approaches to applying LLMs to data analytics that can be implemented today:\n\nLLM writes an analytic code\nLLM writes an analytic subroutine\nUse LLM in an analytic subroutine\n\nWhile these three approaches are not an exhaustive list of how LLMs can be applied to data, they can be easily understood and implemented with Ibis and Marvin in a few lines of code. Together with these two open-source tools, we can build a natural language interface for data analytics that supports 18+ backends.\nBut first, let’s demonstrate the three approaches.\n\nApproach 1: LLM writes analytic code\nState of the art (SoTA) LLMs are decent at generating SQL out of the box. We can be clever to handle errors, retries, and more, but in its simplest form:\n\n\nCode\n1import ibis\nimport marvin\n\nfrom rich import print\nfrom time import sleep\nfrom dotenv import load_dotenv\n\n2load_dotenv()\n\n3con = ibis.connect(\"duckdb://penguins.ddb\")\nt = ibis.examples.penguins.fetch()\nt = con.create_table(\"penguins\", t.to_pyarrow(), overwrite=True)\n\n\n\n1\n\nImport the libraries we need.\n\n2\n\nLoad the environment variable to setup Marvin to call our OpenAI account.\n\n3\n\nSetup the demo datain an Ibis backend.\n\n\n\n\n\n1import ibis\nimport marvin\n\nfrom ibis.expr.schema import Schema\nfrom ibis.expr.types.relations import Table\n\n\n2ibis.options.interactive = True\nmarvin.settings.llm_model = \"openai/gpt-4\"\n\n\n1\n\nImport Ibis and Marvin.\n\n2\n\nConfigure Ibis and Marvin\n\n\n\n\n\n1@marvin.ai_fn\ndef _generate_sql_select(\n    text: str, table_name: str, table_schema: Schema\n) -&gt; str:\n    \"\"\"Generate SQL SELECT from text.\"\"\"\n\n\n2def sql_from_text(text: str, t: Table) -&gt; Table:\n    \"\"\"Run SQL from text.\"\"\"\n    return t.sql(_generate_sql_select(text, t.get_name(), t.schema()).strip(\";\"))\n\n\n1\n\nA non-deterministic, LLM-powered AI function.\n\n2\n\nA deterministic, human-authored function that calls the AI function.\n\n\n\n\n\nt2 = sql_from_text(\"the unique combination of species and islands\", t)\nt2\n\n┏━━━━━━━━━━━┳━━━━━━━━━━━┓\n┃ species   ┃ island    ┃\n┡━━━━━━━━━━━╇━━━━━━━━━━━┩\n│ string    │ string    │\n├───────────┼───────────┤\n│ Adelie    │ Torgersen │\n│ Adelie    │ Biscoe    │\n│ Adelie    │ Dream     │\n│ Gentoo    │ Biscoe    │\n│ Chinstrap │ Dream     │\n└───────────┴───────────┘\n\n\n\n\n\nCode\n1sleep(3)\n\n\n\n1\n\nAvoid rate-limiting by waiting.\n\n\n\n\n\nt3 = sql_from_text(\n    \"the unique combination of species and islands, with their counts, ordered from highest to lowest, and name that column just 'count'\",\n    t,\n)\nt3\n\n┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━┓\n┃ species   ┃ island    ┃ count ┃\n┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━┩\n│ string    │ string    │ int64 │\n├───────────┼───────────┼───────┤\n│ Gentoo    │ Biscoe    │   124 │\n│ Chinstrap │ Dream     │    68 │\n│ Adelie    │ Dream     │    56 │\n│ Adelie    │ Torgersen │    52 │\n│ Adelie    │ Biscoe    │    44 │\n└───────────┴───────────┴───────┘\n\n\n\n\n\nCode\n1sleep(3)\n\n\n\n1\n\nAvoid rate-limiting by waiting.\n\n\n\n\nThis works well-enough for simple cases and can be expanded to handle complex ones. In many scenarios, it may be easier to express a query in English or another language than to write it in SQL, especially if working across multiple SQL dialects.\nSQL isn’t standard, with many dialects across data platforms. Ibis works around this by providing a standard Python API for analytic code but must make compromises to support many data platforms, often via SQL in their native dialect. Substrait is a newer project that aims to solve this problem by providing a standard, portable, and extensible intermediary representation (IR) for data transformation code that Ibis and data platforms could all standardize on. Substrait is still in the early stages of development, but it’s worth keeping an eye on and will be adopted in Ibis once supported across many data platforms.\nFor now, we’ll focus on generating SQL and Python analytical code with LLMs.\n\n\nApproach 2: LLM writes an analytical subroutine\nIf more complex logic needs to be expressed, SoTA LLMs are also decent at writing Python and a number of other programming languages that are used in analytical subroutines. Many data platforms support user-defined functions (UDFs) in Python or some other language. We’ll stick to scalar Python UDFs via DuckDB to demonstrate the concept:\n\n1@marvin.ai_fn\ndef _generate_python_function(text: str) -&gt; str:\n    \"\"\"Generate a simple, typed, correct Python function from text.\"\"\"\n\n\n2def create_udf_from_text(text: str) -&gt; str:\n    \"\"\"Create a UDF from text.\"\"\"\n    return f\"\"\"\nimport ibis\n\n@ibis.udf.scalar.python\n{_generate_python_function(text)}\n\"\"\".strip()\n\n\n1\n\nA non-deterministic, LLM-powered AI function.\n\n2\n\nA deterministic, human-authored function that calls the AI function.\n\n\n\n\n\nudf = create_udf_from_text(\n    \"a function named count_vowels that given an input string, returns an int w/ the number of vowels (y_included as a boolean option defaulted to False)\"\n)\nprint(udf)\nexec(udf)\n\nimport ibis\n\n@ibis.udf.scalar.python\ndef count_vowels(input_string: str, y_included: bool = False) -&gt; int:\n    \"\"\"Given an input string, it returns the number of vowels. y can be included as a vowel.\"\"\"\n    vowels = 'aeiou'\n    if y_included:\n        vowels += 'y'\n    return sum(1 for char in input_string if char in vowels)\n\n\n\n\n\nCode\n1sleep(3)\n\n\n\n1\n\nAvoid rate-limiting by waiting.\n\n\n\n\n\nt4 = t3.mutate(\n    species_vowel_count=count_vowels(t3.species),\n    island_vowel_count=count_vowels(t3.island),\n)\nt4\n\n┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┓\n┃ species   ┃ island    ┃ count ┃ species_vowel_count ┃ island_vowel_count ┃\n┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━┩\n│ string    │ string    │ int64 │ int64               │ int64              │\n├───────────┼───────────┼───────┼─────────────────────┼────────────────────┤\n│ Gentoo    │ Biscoe    │   124 │                   3 │                  3 │\n│ Chinstrap │ Dream     │    68 │                   2 │                  2 │\n│ Adelie    │ Dream     │    56 │                   3 │                  2 │\n│ Adelie    │ Torgersen │    52 │                   3 │                  3 │\n│ Adelie    │ Biscoe    │    44 │                   3 │                  3 │\n└───────────┴───────────┴───────┴─────────────────────┴────────────────────┘\n\n\n\n\n\nCode\n1sleep(3)\n\n\n\n1\n\nAvoid rate-limiting by waiting.\n\n\n\n\nIn this case, there’s no reason not to have a human in the loop reviewing the output code and committing it for production use. This could be useful for quick prototyping or, given a box of tools in the form of UDFs, working through a natural language interface.\n\n\nApproach 3: Use LLM in an analytical subroutine\nWe can also call the LLM once-per-row in the table via a subroutine. For variety, we’ll use an AI model instead of an AI function:\n\n1from pydantic import BaseModel, Field\n\n# decrease cost\n2marvin.settings.llm_model = \"openai/gpt-3.5-turbo-16k\"\n\n\n3@marvin.ai_model\nclass VowelCounter(BaseModel):\n    \"\"\"Count vowels in a string.\"\"\"\n\n    include_y: bool = Field(False, description=\"Include 'y' as a vowel.\")\n    # num_a: int = Field(..., description=\"The number of 'a' vowels.\")\n    # num_e: int = Field(..., description=\"The number of 'e' vowels.\")\n    # num_i: int = Field(..., description=\"The number of 'i' vowels.\")\n    # num_o: int = Field(..., description=\"The number of 'o' vowels.\")\n    # num_u: int = Field(..., description=\"The number of 'u' vowels.\")\n    # num_y: int = Field(..., description=\"The number of 'y' vowels.\")\n    num_total: int = Field(..., description=\"The total number of vowels.\")\n\n\n4VowelCounter(\"hello world\")\n\n\n1\n\nAdditional imports for Pydantic.\n\n2\n\nConfigure Marvin to use a cheaper model.\n\n3\n\nA non-deterministic, LLM-powered AI model.\n\n4\n\nCall the AI model on some text.\n\n\n\n\nVowelCounter(include_y=False, num_total=3)\n\n\nThen we’ll have the LLM write the UDF that calls the LLM, just to be fancy:\n\nudf = create_udf_from_text(\n    \"a function named count_vowels_ai that given an input string, calls VowelCounter on it and returns the num_total attribute of that result\"\n)\nprint(udf)\nexec(udf)\n\nimport ibis\n\n@ibis.udf.scalar.python\ndef count_vowels_ai(input_string: str) -&gt; int:\n    result = VowelCounter(input_string)\n    return result.num_total\n\n\n\n\n\nCode\n1sleep(3)\n\n\n\n1\n\nAvoid rate-limiting by waiting.\n\n\n\n\n\nt5 = t3.mutate(\n    species_vowel_count=count_vowels_ai(t3.species),\n    island_vowel_count=count_vowels_ai(t3.island),\n)\nt5\n\n┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┓\n┃ species   ┃ island    ┃ count ┃ species_vowel_count ┃ island_vowel_count ┃\n┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━┩\n│ string    │ string    │ int64 │ int64               │ int64              │\n├───────────┼───────────┼───────┼─────────────────────┼────────────────────┤\n│ Gentoo    │ Biscoe    │   124 │                   3 │                  3 │\n│ Chinstrap │ Dream     │    68 │                   2 │                  1 │\n│ Adelie    │ Dream     │    56 │                   3 │                  2 │\n│ Adelie    │ Torgersen │    52 │                   3 │                  3 │\n│ Adelie    │ Biscoe    │    44 │                   3 │                  3 │\n└───────────┴───────────┴───────┴─────────────────────┴────────────────────┘\n\n\n\nNotice that in this UDF, unlike in the previous example, a LLM is being called (possibly several times) for each row in the table. This is a very expensive operation and we’ll need to be careful about how we use it in practice.\n\n\nCode\n1sleep(3)\n\n\n\n1\n\nAvoid rate-limiting by waiting."
  },
  {
    "objectID": "posts/llms-and-data-pt1/index.html#summary",
    "href": "posts/llms-and-data-pt1/index.html#summary",
    "title": "Three approaches",
    "section": "Summary",
    "text": "Summary\nTo summarize this post:\n\nfrom rich import print\n\nwith open(\"index.qmd\", \"r\") as f:\n    self_text = f.read()\n\n# increease accuracy\nmarvin.settings.llm_model = \"openai/gpt-4\"\n\n@marvin.ai_model\nclass Summary(BaseModel):\n    \"\"\"Summary of text.\"\"\"\n\n    summary_line: str = Field(..., description=\"The one-line summary of the text.\")\n    summary_paragraph: str = Field(\n        ..., description=\"The one-paragraph summary of the text.\"\n    )\n    conclusion: str = Field(\n        ..., description=\"The conclusion the reader should draw from the text.\"\n    )\n    key_points: list[str] = Field(..., description=\"The key points of the text.\")\n    critiques: list[str] = Field(\n        ..., description=\"Professional, fair critiques of the text.\"\n    )\n    suggested_improvements: list[str] = Field(\n        ..., description=\"Suggested improvements for the text.\"\n    )\n    sentiment: float = Field(..., description=\"The sentiment of the text.\")\n    sentiment_label: str = Field(..., description=\"The sentiment label of the text.\")\n    author_bias: str = Field(..., description=\"The author bias of the text.\")\n\n\nprint(Summary(self_text))\n\nSummary(\n    summary_line=\"The blog post titled 'Three approaches' by Cody Peterson, dated 2023-10-13, discusses three \ndistinct approaches to applying Language Learning Models (LLMs) to data analytics using open-source tools Ibis and \nMarvin.\",\n    summary_paragraph='The author explains three approaches to applying LLMs to data analytics. The first approach \nis letting the LLM write an analytic code, specifically generating SQL. The second approach involves letting LLM \nwrite an analytic subroutine, demonstrated by writing Python code for user-defined functions. The third approach \nuses LLM in an analytic subroutine, illustrated by calling the LLM once-per-row in a table via a subroutine. Each \nmethod has potential applications and limitations, and the choice of approach will depend on the specific \nrequirements of the data analytics task.',\n    conclusion='LLMs can be used to transform and analyze data in various ways. By using these approaches in \ncombination with open-source tools like Ibis and Marvin, it is possible to build a natural language interface for \ndata analytics that supports multiple backends. The author concludes by introducing an open-source data & AI \nproject for building next-generation natural language interfaces to data, Ibis Birdbrain.',\n    key_points=[\n        'The LLM can write an analytic code, particularly generating SQL.',\n        'The LLM can also write an analytic subroutine, exemplified by writing Python code for user-defined \nfunctions.',\n        'The LLM can be used in an analytic subroutine, shown by calling the LLM once-per-row in a table via a \nsubroutine.',\n        'The author suggests that each approach has its potential applications and limitations.',\n        'The choice of approach will depend on the specific requirements of the data analytics task.'\n    ],\n    critiques=[\n        'The author could have provided more concrete examples or case studies to illustrate the application of \nthese approaches.',\n        'A comparison of the advantages and disadvantages of each approach would have been helpful.'\n    ],\n    suggested_improvements=[\n        'The author could include real-world applications or examples of each approach.',\n        'A detailed comparison of the pros and cons of each approach would provide better guidance for readers.'\n    ],\n    sentiment=0.25,\n    sentiment_label='Neutral',\n    author_bias='The author shows a positive bias towards the use of open-source tools, Ibis and Marvin, for data \nanalytics.'\n)"
  },
  {
    "objectID": "posts/llms-and-data-pt1/index.html#next-steps",
    "href": "posts/llms-and-data-pt1/index.html#next-steps",
    "title": "Three approaches",
    "section": "Next steps",
    "text": "Next steps\nYou can get involved with Ibis Birdbrain, our open-source data & AI project for building next-generation natural language interfaces to data.\nRead the next post in this series."
  },
  {
    "objectID": "posts/llms-and-data-pt3/index.html",
    "href": "posts/llms-and-data-pt3/index.html",
    "title": "Composable data + AI systems",
    "section": "",
    "text": "When writing (data and AI) code, it’s important to use the right abstraction for your job to be done. A modular and composable data and AI system is easier than ever with the new generation of LLM-powered tools, but lacks a standard, modular, interoperable, customizable, and extensible framework. Can Ibis Birdbrain be that standard?"
  },
  {
    "objectID": "posts/llms-and-data-pt3/index.html#introduction",
    "href": "posts/llms-and-data-pt3/index.html#introduction",
    "title": "Composable data + AI systems",
    "section": "",
    "text": "When writing (data and AI) code, it’s important to use the right abstraction for your job to be done. A modular and composable data and AI system is easier than ever with the new generation of LLM-powered tools, but lacks a standard, modular, interoperable, customizable, and extensible framework. Can Ibis Birdbrain be that standard?"
  },
  {
    "objectID": "posts/llms-and-data-pt3/index.html#standards",
    "href": "posts/llms-and-data-pt3/index.html#standards",
    "title": "Composable data + AI systems",
    "section": "Standards",
    "text": "Standards\n\n\nWhen things don’t work as they should, it often means that standards are absent.\n- The International Organization for Standardization (ISO)\n\n- The Composable Codex\n\nThe standard that has emerged from the recent LLM applications is the power of a conversation in interacting with a computer. If we can give that interaction tht ability to interact with data platforms, we can achieve a lot through natural language."
  },
  {
    "objectID": "posts/llms-and-data-pt3/index.html#composable-data-and-ai-systems-are-mice",
    "href": "posts/llms-and-data-pt3/index.html#composable-data-and-ai-systems-are-mice",
    "title": "Composable data + AI systems",
    "section": "Composable data (and AI) systems are MICE",
    "text": "Composable data (and AI) systems are MICE\nComposable data systems are MICE: modular, interoperable, customizable, and extensible.\nWith any new tech, there’s a lot of duplication of effort and reinvention of wheels. Ibis Birdbrain aims to be an open-source standard for building data and AI systems in Python that adheres to MICE best practices and works with any data and AI platform. It achieves this by building on top of Ibis and Marvin, handling calls to the data and AI platforms respectively. Ibis and Marvin are themselves built on many other open-source projects, but provide the right level of abstraction for building composable data and AI systems, independently. Ibis Birdbrain glues them together enables a natural language interface that’s interoperable between 18+ data platforms and any AI platform Marvin supports.\nglue that enables working via natural language with any data platform Ibis supports and any AI platform Marvin supports.\n\n\n\n\n\n\nImportant\n\n\n\nIbis Birdbrain is very early in development and looking for community input and contributors. We believe it’s important for there to be a standard, and I can see Ibis Birdbrain going in several directions:\n\nA useful template for building a language user interface with data tools\nA standard for building data and AI systems with MICE components\n\nIf you’re interested in either of these directions, please get involved with the project!\nIbis Birdbrain is under Ibis project governance."
  },
  {
    "objectID": "posts/llms-and-data-pt3/index.html#looking-ahead",
    "href": "posts/llms-and-data-pt3/index.html#looking-ahead",
    "title": "Composable data + AI systems",
    "section": "Looking ahead",
    "text": "Looking ahead\nI am not a software engineer, I just saw a cool application of Ibis and Marvin. The objective of releasing Ibis Birdbrain as an open-source project under Ibis governance is to enable the community to build a standard for composable data and AI systems. My hope is for the Ibis, Marvin, and broader data/AI Python communities to build the standard in the open, and properietary applications on top of those standards. This is important for a composable data ecosystem that can be modular, interoperable, customizable, and extensible."
  },
  {
    "objectID": "posts/llms-and-data-pt3/index.html#how-is-ibis-birdbrain-mice-now-whats-next",
    "href": "posts/llms-and-data-pt3/index.html#how-is-ibis-birdbrain-mice-now-whats-next",
    "title": "Composable data + AI systems",
    "section": "How is Ibis Birdbrain MICE now? What’s next?",
    "text": "How is Ibis Birdbrain MICE now? What’s next?"
  },
  {
    "objectID": "posts/llms-and-data-pt3/index.html#next-steps",
    "href": "posts/llms-and-data-pt3/index.html#next-steps",
    "title": "Composable data + AI systems",
    "section": "Next steps",
    "text": "Next steps\nYou can get involved with Ibis Birdbrain, our open-source data & AI project for building next-generation natural language interfaces to data.\nRead the next post in this series, building Ibis Birdbrain."
  },
  {
    "objectID": "demo.html",
    "href": "demo.html",
    "title": "demo",
    "section": "",
    "text": "Back to top"
  },
  {
    "objectID": "index.html",
    "href": "index.html",
    "title": "Ibis Birdbrain",
    "section": "",
    "text": "Why Ibis Birdbrain?\n  \n  \n    \n     Installation\n  \n  \n    \n     Tutorial: getting started\n  \n  \n    \n     GitHub\n  \n  \n    \n     Chat\n  \n\n\n\n\n\n\n\n\n\nWarning\n\n\n\nIbis Birdbrain is highly experimental and currently under construction by slow humans. Please check back soon!\n\n\n\n\n\n\n Back to top"
  },
  {
    "objectID": "how-to/input-output/basics.html",
    "href": "how-to/input-output/basics.html",
    "title": "Basic input/output",
    "section": "",
    "text": "You can…",
    "crumbs": [
      "Input Output",
      "Basic input/output"
    ]
  },
  {
    "objectID": "concepts/llms.html",
    "href": "concepts/llms.html",
    "title": "Large language models",
    "section": "",
    "text": "Large language models (LLMs) represent decades of research and development of neural networks. While relatively impressive LLMs have been around for years, recent innovations have made it possible to create instruction-following, conversational bots that can perform tasks on behalf of the user.\nWe are primarily concerned with applying LLMs to data, but we’ll take a brief look at how they work and why we should use them.\n\n\nAn artificial neural network (ANN or often just NN) is a computational model that is loosely inspired by the biological neural networks in the brain. It is a collection of connected nodes, called neurons, that are organized into layers. Each neuron is connected to other neurons in the network, and each connection has a weight associated with it. The weights are adjusted during training to improve the model’s performance.\nAn instance of a neural network (and many other ML architectures) is called a model. A model has usually been trained on data to learn to represent a system. While they are amny machine learning model architectures and training algorithms, the fundamental innovation of (large/deep) neural networks is the ability to represent an arbitrary system.\n\n\n\nA large language model is a neural network trained on vast amounts of text data.\n\n\n\nA LLM takes text as input and produces text as output.\n\n\n\nText in, text out. Neural networks and LLMs by design are non-determinstic. Though there are many tricks and workarounds, relying on LLMs for determinstic behavior is a bad idea. Instead, LLMs are great for:\n\ntext-based ML tasks (like classification, clustering)\ntext-based entity extraction (named entity regognition)\ntext-based generation (like summarization, translation, and question answering)\nother text-based tasks\n\nLLMs today are decent, but flawed, at generating programming code (as text). We can again use clever tricks and program around the non-determinstic behavior (such as running code, checking for any errors, and making one or more attempts via LLM to the errors). Fundamentally, keep in mind that an input to LLM is always text and an output is always text.\n\n\n\nSome limitations include:\n\ncost\nlatency\naccuracy\n…",
    "crumbs": [
      "Large language models"
    ]
  },
  {
    "objectID": "concepts/llms.html#what-is-an-artificial-neural-network",
    "href": "concepts/llms.html#what-is-an-artificial-neural-network",
    "title": "Large language models",
    "section": "",
    "text": "An artificial neural network (ANN or often just NN) is a computational model that is loosely inspired by the biological neural networks in the brain. It is a collection of connected nodes, called neurons, that are organized into layers. Each neuron is connected to other neurons in the network, and each connection has a weight associated with it. The weights are adjusted during training to improve the model’s performance.\nAn instance of a neural network (and many other ML architectures) is called a model. A model has usually been trained on data to learn to represent a system. While they are amny machine learning model architectures and training algorithms, the fundamental innovation of (large/deep) neural networks is the ability to represent an arbitrary system.",
    "crumbs": [
      "Large language models"
    ]
  },
  {
    "objectID": "concepts/llms.html#what-is-a-large-language-model",
    "href": "concepts/llms.html#what-is-a-large-language-model",
    "title": "Large language models",
    "section": "",
    "text": "A large language model is a neural network trained on vast amounts of text data.",
    "crumbs": [
      "Large language models"
    ]
  },
  {
    "objectID": "concepts/llms.html#what-are-the-inputs-and-outputs",
    "href": "concepts/llms.html#what-are-the-inputs-and-outputs",
    "title": "Large language models",
    "section": "",
    "text": "A LLM takes text as input and produces text as output.",
    "crumbs": [
      "Large language models"
    ]
  },
  {
    "objectID": "concepts/llms.html#what-do-llms-work-well-for",
    "href": "concepts/llms.html#what-do-llms-work-well-for",
    "title": "Large language models",
    "section": "",
    "text": "Text in, text out. Neural networks and LLMs by design are non-determinstic. Though there are many tricks and workarounds, relying on LLMs for determinstic behavior is a bad idea. Instead, LLMs are great for:\n\ntext-based ML tasks (like classification, clustering)\ntext-based entity extraction (named entity regognition)\ntext-based generation (like summarization, translation, and question answering)\nother text-based tasks\n\nLLMs today are decent, but flawed, at generating programming code (as text). We can again use clever tricks and program around the non-determinstic behavior (such as running code, checking for any errors, and making one or more attempts via LLM to the errors). Fundamentally, keep in mind that an input to LLM is always text and an output is always text.",
    "crumbs": [
      "Large language models"
    ]
  },
  {
    "objectID": "concepts/llms.html#what-are-the-limitations-and-considerations",
    "href": "concepts/llms.html#what-are-the-limitations-and-considerations",
    "title": "Large language models",
    "section": "",
    "text": "Some limitations include:\n\ncost\nlatency\naccuracy\n…",
    "crumbs": [
      "Large language models"
    ]
  },
  {
    "objectID": "concepts/user-interfaces.html",
    "href": "concepts/user-interfaces.html",
    "title": "User interfaces",
    "section": "",
    "text": "A user interface (UI) in our context is the mechanism by which a human interacts with software.\n\n\nA command-line interface (CLI) is a UI for interacting with a computer program where the user (or client) issues commands to the program in the form of successive lines of text (command lines). CLIs are great for automating tasks and often user-friendly, with a short learning curve (if designed well).\nCLIs are great for containing common data analysis tasks expressed in a single, configurable command. However, remembering the exact syntax of a command can be difficult, especially if the command is not used frequently.\n\n\n\nPython can be an interactive UI in a terminal, notebook, IDE, or other setup to serve as a user-interface for data analytics.\n\n\n\nIbis Birdbrain aims to implement a language user interface (LUI) for data analytics, providing additional AI assistance in both a CLI and Python interface. This enables users to interact with a bot through natural language that can perform tasks on their behalf.\n\n\n\n\n\n\nWarning\n\n\n\nThis LUI paradigm is highly experimental and should be put in production (automated) with caution.",
    "crumbs": [
      "User interfaces"
    ]
  },
  {
    "objectID": "concepts/user-interfaces.html#command-line-interface",
    "href": "concepts/user-interfaces.html#command-line-interface",
    "title": "User interfaces",
    "section": "",
    "text": "A command-line interface (CLI) is a UI for interacting with a computer program where the user (or client) issues commands to the program in the form of successive lines of text (command lines). CLIs are great for automating tasks and often user-friendly, with a short learning curve (if designed well).\nCLIs are great for containing common data analysis tasks expressed in a single, configurable command. However, remembering the exact syntax of a command can be difficult, especially if the command is not used frequently.",
    "crumbs": [
      "User interfaces"
    ]
  },
  {
    "objectID": "concepts/user-interfaces.html#python",
    "href": "concepts/user-interfaces.html#python",
    "title": "User interfaces",
    "section": "",
    "text": "Python can be an interactive UI in a terminal, notebook, IDE, or other setup to serve as a user-interface for data analytics.",
    "crumbs": [
      "User interfaces"
    ]
  },
  {
    "objectID": "concepts/user-interfaces.html#language-user-interface",
    "href": "concepts/user-interfaces.html#language-user-interface",
    "title": "User interfaces",
    "section": "",
    "text": "Ibis Birdbrain aims to implement a language user interface (LUI) for data analytics, providing additional AI assistance in both a CLI and Python interface. This enables users to interact with a bot through natural language that can perform tasks on their behalf.\n\n\n\n\n\n\nWarning\n\n\n\nThis LUI paradigm is highly experimental and should be put in production (automated) with caution.",
    "crumbs": [
      "User interfaces"
    ]
  },
  {
    "objectID": "release_notes.html",
    "href": "release_notes.html",
    "title": "Release notes",
    "section": "",
    "text": "Release notes\nTBD\n\n\n\n\n Back to top"
  },
  {
    "objectID": "tutorials/cli.html",
    "href": "tutorials/cli.html",
    "title": "Tutorial: CLI",
    "section": "",
    "text": "Install Ibis Birdbrain\n\n\n\n\nWith Ibis Birdbrain installed, you can use the birdbrain command-line interface (CLI):\n$ birdbrain\n\n\n\nLearn how to work with Ibis Birdbrain in Python.",
    "crumbs": [
      "Tutorial: CLI"
    ]
  },
  {
    "objectID": "tutorials/cli.html#prerequisites",
    "href": "tutorials/cli.html#prerequisites",
    "title": "Tutorial: CLI",
    "section": "",
    "text": "Install Ibis Birdbrain",
    "crumbs": [
      "Tutorial: CLI"
    ]
  },
  {
    "objectID": "tutorials/cli.html#overview",
    "href": "tutorials/cli.html#overview",
    "title": "Tutorial: CLI",
    "section": "",
    "text": "With Ibis Birdbrain installed, you can use the birdbrain command-line interface (CLI):\n$ birdbrain",
    "crumbs": [
      "Tutorial: CLI"
    ]
  },
  {
    "objectID": "tutorials/cli.html#starting-an-interactive-python-session",
    "href": "tutorials/cli.html#starting-an-interactive-python-session",
    "title": "Tutorial: CLI",
    "section": "",
    "text": "You can use the ipy subcommand to start an interactive Python session with Ibis Birdbrain ready to use:\n$ birdbrain ipy\naccess to: birdbrain\nmodel: azure_openai/gpt-4-32k\nPython 3.11.5 (main, Sep 14 2023, 13:17:51) [Clang 14.0.3 (clang-1403.0.22.14.1)]\nType 'copyright', 'credits' or 'license' for more information\nIPython 8.16.0 -- An enhanced Interactive Python. Type '?' for help.\n\n[ins] In [1]: birdbrain\nOut[1]: &lt;Bot: birdbrain&gt;\n\n[ins] In [2]:",
    "crumbs": [
      "Tutorial: CLI"
    ]
  },
  {
    "objectID": "tutorials/cli.html#next-steps",
    "href": "tutorials/cli.html#next-steps",
    "title": "Tutorial: CLI",
    "section": "",
    "text": "Learn how to work with Ibis Birdbrain in Python.",
    "crumbs": [
      "Tutorial: CLI"
    ]
  },
  {
    "objectID": "concepts/bots.html",
    "href": "concepts/bots.html",
    "title": "Bots",
    "section": "",
    "text": "Ibis Birdbrain implements a Bot calss that can be used to instantiate one or more bots that automate various tasks.",
    "crumbs": [
      "Bots"
    ]
  },
  {
    "objectID": "concepts/bots.html#usage",
    "href": "concepts/bots.html#usage",
    "title": "Bots",
    "section": "Usage",
    "text": "Usage\n\nfrom ibis_birdbrain import Bot\nBot\n\nibis_birdbrain.bot.Bot",
    "crumbs": [
      "Bots"
    ]
  },
  {
    "objectID": "concepts/archive/user-interfaces.html",
    "href": "concepts/archive/user-interfaces.html",
    "title": "User interfaces",
    "section": "",
    "text": "A user interface (UI) in our context is the mechanism by which a human interacts with software.\n\n\nA command-line interface (CLI) is a UI for interacting with a computer program where the user (or client) issues commands to the program in the form of successive lines of text (command lines). CLIs are great for automating tasks and often user-friendly, with a short learning curve (if designed well).\nCLIs are great for containing common data analysis tasks expressed in a single, configurable command. However, remembering the exact syntax of a command can be difficult, especially if the command is not used frequently.\n\n\n\nPython can be an interactive UI in a terminal, notebook, IDE, or other setup to serve as a user-interface for data analytics.\n\n\n\nIbis Birdbrain aims to implement a language user interface (LUI) for data analytics, providing additional AI assistance in both a CLI and Python interface. This enables users to interact with a bot through natural language that can perform tasks on their behalf.\n\n\n\n\n\n\nWarning\n\n\n\nThis LUI paradigm is highly experimental and should be put in production (automated) with caution."
  },
  {
    "objectID": "concepts/archive/user-interfaces.html#command-line-interface",
    "href": "concepts/archive/user-interfaces.html#command-line-interface",
    "title": "User interfaces",
    "section": "",
    "text": "A command-line interface (CLI) is a UI for interacting with a computer program where the user (or client) issues commands to the program in the form of successive lines of text (command lines). CLIs are great for automating tasks and often user-friendly, with a short learning curve (if designed well).\nCLIs are great for containing common data analysis tasks expressed in a single, configurable command. However, remembering the exact syntax of a command can be difficult, especially if the command is not used frequently."
  },
  {
    "objectID": "concepts/archive/user-interfaces.html#python",
    "href": "concepts/archive/user-interfaces.html#python",
    "title": "User interfaces",
    "section": "",
    "text": "Python can be an interactive UI in a terminal, notebook, IDE, or other setup to serve as a user-interface for data analytics."
  },
  {
    "objectID": "concepts/archive/user-interfaces.html#language-user-interface",
    "href": "concepts/archive/user-interfaces.html#language-user-interface",
    "title": "User interfaces",
    "section": "",
    "text": "Ibis Birdbrain aims to implement a language user interface (LUI) for data analytics, providing additional AI assistance in both a CLI and Python interface. This enables users to interact with a bot through natural language that can perform tasks on their behalf.\n\n\n\n\n\n\nWarning\n\n\n\nThis LUI paradigm is highly experimental and should be put in production (automated) with caution."
  },
  {
    "objectID": "concepts/archive/llms.html",
    "href": "concepts/archive/llms.html",
    "title": "Large language models",
    "section": "",
    "text": "Large language models (LLMs) represent decades of research and development of neural networks. While relatively impressive LLMs have been around for years, recent innovations have made it possible to create instruction-following, conversational bots that can perform tasks on behalf of the user.\nWe are primarily concerned with applying LLMs to data, but we’ll take a brief look at how they work and why we should use them.\n\n\nAn artificial neural network (ANN or often just NN) is a computational model that is loosely inspired by the biological neural networks in the brain. It is a collection of connected nodes, called neurons, that are organized into layers. Each neuron is connected to other neurons in the network, and each connection has a weight associated with it. The weights are adjusted during training to improve the model’s performance.\nAn instance of a neural network (and many other ML architectures) is called a model. A model has usually been trained on data to learn to represent a system. While they are amny machine learning model architectures and training algorithms, the fundamental innovation of (large/deep) neural networks is the ability to represent an arbitrary system.\n\n\n\nA large language model is a neural network trained on vast amounts of text data.\n\n\n\nA LLM takes text as input and produces text as output.\n\n\n\nText in, text out. Neural networks and LLMs by design are non-determinstic. Though there are many tricks and workarounds, relying on LLMs for determinstic behavior is a bad idea. Instead, LLMs are great for:\n\ntext-based ML tasks (like classification, clustering)\ntext-based entity extraction (named entity regognition)\ntext-based generation (like summarization, translation, and question answering)\nother text-based tasks\n\nLLMs today are decent, but flawed, at generating programming code (as text). We can again use clever tricks and program around the non-determinstic behavior (such as running code, checking for any errors, and making one or more attempts via LLM to the errors). Fundamentally, keep in mind that an input to LLM is always text and an output is always text.\n\n\n\nSome limitations include:\n\ncost\nlatency\naccuracy\n…"
  },
  {
    "objectID": "concepts/archive/llms.html#what-is-an-artificial-neural-network",
    "href": "concepts/archive/llms.html#what-is-an-artificial-neural-network",
    "title": "Large language models",
    "section": "",
    "text": "An artificial neural network (ANN or often just NN) is a computational model that is loosely inspired by the biological neural networks in the brain. It is a collection of connected nodes, called neurons, that are organized into layers. Each neuron is connected to other neurons in the network, and each connection has a weight associated with it. The weights are adjusted during training to improve the model’s performance.\nAn instance of a neural network (and many other ML architectures) is called a model. A model has usually been trained on data to learn to represent a system. While they are amny machine learning model architectures and training algorithms, the fundamental innovation of (large/deep) neural networks is the ability to represent an arbitrary system."
  },
  {
    "objectID": "concepts/archive/llms.html#what-is-a-large-language-model",
    "href": "concepts/archive/llms.html#what-is-a-large-language-model",
    "title": "Large language models",
    "section": "",
    "text": "A large language model is a neural network trained on vast amounts of text data."
  },
  {
    "objectID": "concepts/archive/llms.html#what-are-the-inputs-and-outputs",
    "href": "concepts/archive/llms.html#what-are-the-inputs-and-outputs",
    "title": "Large language models",
    "section": "",
    "text": "A LLM takes text as input and produces text as output."
  },
  {
    "objectID": "concepts/archive/llms.html#what-do-llms-work-well-for",
    "href": "concepts/archive/llms.html#what-do-llms-work-well-for",
    "title": "Large language models",
    "section": "",
    "text": "Text in, text out. Neural networks and LLMs by design are non-determinstic. Though there are many tricks and workarounds, relying on LLMs for determinstic behavior is a bad idea. Instead, LLMs are great for:\n\ntext-based ML tasks (like classification, clustering)\ntext-based entity extraction (named entity regognition)\ntext-based generation (like summarization, translation, and question answering)\nother text-based tasks\n\nLLMs today are decent, but flawed, at generating programming code (as text). We can again use clever tricks and program around the non-determinstic behavior (such as running code, checking for any errors, and making one or more attempts via LLM to the errors). Fundamentally, keep in mind that an input to LLM is always text and an output is always text."
  },
  {
    "objectID": "concepts/archive/llms.html#what-are-the-limitations-and-considerations",
    "href": "concepts/archive/llms.html#what-are-the-limitations-and-considerations",
    "title": "Large language models",
    "section": "",
    "text": "Some limitations include:\n\ncost\nlatency\naccuracy\n…"
  },
  {
    "objectID": "concepts/archive/platforms.html",
    "href": "concepts/archive/platforms.html",
    "title": "Data and AI platforms",
    "section": "",
    "text": "Data and AI platforms are…"
  },
  {
    "objectID": "concepts/archive/ops.html",
    "href": "concepts/archive/ops.html",
    "title": "LLMOps, MLOps, DevOps",
    "section": "",
    "text": "The ML landscape is plagued by “the toy problem” – building something cool is easy, putting something in production is hard."
  },
  {
    "objectID": "concepts/messages.html",
    "href": "concepts/messages.html",
    "title": "Messages",
    "section": "",
    "text": "Ibis Birdbrain communicates with the user, itself, and (eventually) other bots through Messages. A Message is a simple wrapper around text with metadata and optional Attachments.",
    "crumbs": [
      "Messages"
    ]
  },
  {
    "objectID": "concepts/messages.html#usage",
    "href": "concepts/messages.html#usage",
    "title": "Messages",
    "section": "Usage",
    "text": "Usage\n\nfrom ibis_birdbrain.messages import Message, Messages, Email\n\nm1 = Message(\"Hello, world!\")\nm1\n\nMessage(df8f6f11-7362-47d8-b932-62424d3cba8d)",
    "crumbs": [
      "Messages"
    ]
  },
  {
    "objectID": "concepts/messages.html#emails",
    "href": "concepts/messages.html#emails",
    "title": "Messages",
    "section": "Emails",
    "text": "Emails\nCurrently, the only implementation of Message that is viewable as a proper string is Email.\n\ne1 = Email(\"Hello\")\ne2 = Email(\", world!\")\n\nmessages = Messages([e1, e2])\nmessages\n\nTo: \nFrom: \nSubject: \nSent at: 2024-03-05 09:03:08.698844\nMessage: adca7673-ec5c-4377-bd33-b933fb423af7\n\nHello\n\nAttachments:\n\n\n---\nTo: \nFrom: \nSubject: \nSent at: 2024-03-05 09:03:08.698872\nMessage: 22619ca0-0390-4eaf-951d-459ad636a13f\n\n, world!\n\nAttachments:",
    "crumbs": [
      "Messages"
    ]
  },
  {
    "objectID": "concepts/attachments.html",
    "href": "concepts/attachments.html",
    "title": "Attachments",
    "section": "",
    "text": "Ibis Birdbrain passes Python objects as Attachments to Messages. This allows the user, itself, and (eventually) other bots to interact with data, code, and more.",
    "crumbs": [
      "Attachments"
    ]
  },
  {
    "objectID": "concepts/attachments.html#usage",
    "href": "concepts/attachments.html#usage",
    "title": "Attachments",
    "section": "Usage",
    "text": "Usage\n\nfrom ibis_birdbrain.attachments import Attachment, Attachments\n\na1 = Attachment(content=\"Hello, world!\")\na1\n\nAttachment\n    **guid**: 80dffb16-2493-49cd-ba74-819512179b32\n    **time**: 2024-03-05 09:07:29.230828\n    **name**: None\n    **desc**: None",
    "crumbs": [
      "Attachments"
    ]
  },
  {
    "objectID": "concepts/attachments.html#emails",
    "href": "concepts/attachments.html#emails",
    "title": "Messages",
    "section": "Emails",
    "text": "Emails\nCurrently, the only implementation of Message that is viewable as a proper string is Email.\n\ne1 = Email(\"Hello\")\ne2 = Email(\", world!\")\n\nmessages = Messages([e1, e2])\nmessages\n\nTo: \nFrom: \nSubject: \nSent at: 2024-03-05 09:03:40.756829\nMessage: b9ef0727-a8f7-4724-998f-f97a7dee21c9\n\nHello\n\nAttachments:\n\n\n---\nTo: \nFrom: \nSubject: \nSent at: 2024-03-05 09:03:40.756857\nMessage: 31ceb51d-2c54-4879-ab64-fd4d28f9afbb\n\n, world!\n\nAttachments:",
    "crumbs": [
      "Messages"
    ]
  },
  {
    "objectID": "concepts/attachments.html#tableattachment",
    "href": "concepts/attachments.html#tableattachment",
    "title": "Attachments",
    "section": "TableAttachment",
    "text": "TableAttachment\nA TableAttachment contains an Ibis table:\n\nimport ibis\n\nfrom ibis_birdbrain.attachments import TableAttachment\n\nibis.options.interactive = True\n\nt = ibis.examples.penguins.fetch()\n\na2 = TableAttachment(content=t)\na2\n\nINFO:pins.cache:cache file: /Users/cody/Library/Caches/pins-py/gcs_332a30997e141da0e08f15fbfae8b3c3ec90463922d117a96fa3b1bef85a2a4c/penguins/20230905T090411Z-9aae2/data.txt\nINFO:pins.cache:cache file: /Users/cody/Library/Caches/pins-py/gcs_332a30997e141da0e08f15fbfae8b3c3ec90463922d117a96fa3b1bef85a2a4c/penguins/20230905T090411Z-9aae2/penguins.csv.gz\n\n\n\n\n\n\nTableAttachment\n    **guid**: 95374920-3a86-4287-86dd-7e33e2410091\n    **time**: 2024-03-05 09:07:29.724794\n    **name**: penguins\n    **desc**: \nibis.Schema {\n  species            string\n  island             string\n  bill_length_mm     float64\n  bill_depth_mm      float64\n  flipper_length_mm  int64\n  body_mass_g        int64\n  sex                string\n  year               int64\n}\n                **table**:\n┏━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┓\n┃ species ┃ island    ┃ bill_length_mm ┃ bill_depth_mm ┃ flipper_length_mm ┃ body_mass_g ┃ sex    ┃ year  ┃\n┡━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━┩\n│ string  │ string    │ float64        │ float64       │ int64             │ int64       │ string │ int64 │\n├─────────┼───────────┼────────────────┼───────────────┼───────────────────┼─────────────┼────────┼───────┤\n│ Adelie  │ Torgersen │           39.1 │          18.7 │               181 │        3750 │ male   │  2007 │\n│ Adelie  │ Torgersen │           39.5 │          17.4 │               186 │        3800 │ female │  2007 │\n│ Adelie  │ Torgersen │           40.3 │          18.0 │               195 │        3250 │ female │  2007 │\n│ Adelie  │ Torgersen │           NULL │          NULL │              NULL │        NULL │ NULL   │  2007 │\n│ Adelie  │ Torgersen │           36.7 │          19.3 │               193 │        3450 │ female │  2007 │\n│ Adelie  │ Torgersen │           39.3 │          20.6 │               190 │        3650 │ male   │  2007 │\n│ Adelie  │ Torgersen │           38.9 │          17.8 │               181 │        3625 │ female │  2007 │\n│ Adelie  │ Torgersen │           39.2 │          19.6 │               195 │        4675 │ male   │  2007 │\n│ Adelie  │ Torgersen │           34.1 │          18.1 │               193 │        3475 │ NULL   │  2007 │\n│ Adelie  │ Torgersen │           42.0 │          20.2 │               190 │        4250 │ NULL   │  2007 │\n│ …       │ …         │              … │             … │                 … │           … │ …      │     … │\n└─────────┴───────────┴────────────────┴───────────────┴───────────────────┴─────────────┴────────┴───────┘\n\n\n\nNotice the name, description (schema), and preview are automatically populated.",
    "crumbs": [
      "Attachments"
    ]
  },
  {
    "objectID": "concepts/attachments.html#codeattachment",
    "href": "concepts/attachments.html#codeattachment",
    "title": "Attachments",
    "section": "CodeAttachment",
    "text": "CodeAttachment\nA CodeAttachment contains code – typically Python or SQL:\n\nfrom ibis_birdbrain.attachments import CodeAttachment\n\na3 = CodeAttachment(content=\"select 1 as id\", language=\"sql\")\na3\n\nCodeAttachment\n    **guid**: 155834a3-a1c1-4b93-9f71-fea15100e9c1\n    **time**: 2024-03-05 09:07:29.756722\n    **name**: None\n    **desc**: None\n    **language**: sql\n    **code**:\nselect 1 as id",
    "crumbs": [
      "Attachments"
    ]
  },
  {
    "objectID": "concepts/flows.html",
    "href": "concepts/flows.html",
    "title": "Flows",
    "section": "",
    "text": "Ibis Birdbrain’s Bot chooses a Flow to execute based on Messages.\nA Flow takes Messages as input and returns Messages as output. The details of a given Flow are specific to itself, running a series of Tasks to accomplish its goal.",
    "crumbs": [
      "Flows"
    ]
  },
  {
    "objectID": "concepts/flows.html#usage",
    "href": "concepts/flows.html#usage",
    "title": "Flows",
    "section": "Usage",
    "text": "Usage\n\nfrom ibis_birdbrain.flows import Flow, Flows\n\nflow = Flow()\nflow\n\n&lt;ibis_birdbrain.flows.Flow at 0x1072d0510&gt;",
    "crumbs": [
      "Flows"
    ]
  },
  {
    "objectID": "concepts/tasks.html",
    "href": "concepts/tasks.html",
    "title": "Tasks",
    "section": "",
    "text": "Ibis Birdbrain’s Flow executes one or more Tasks to accomplish its goal. A Task is a single unit of work that takes a Message as input and returns a Message as output.",
    "crumbs": [
      "Tasks"
    ]
  },
  {
    "objectID": "concepts/tasks.html#usage",
    "href": "concepts/tasks.html#usage",
    "title": "Tasks",
    "section": "Usage",
    "text": "Usage\n\nfrom ibis_birdbrain.tasks import Task, Tasks\n\ntask = Task()\ntask\n\n&lt;ibis_birdbrain.tasks.Task at 0x105d42f10&gt;",
    "crumbs": [
      "Tasks"
    ]
  },
  {
    "objectID": "why.html#language-models-lms",
    "href": "why.html#language-models-lms",
    "title": "Why Ibis Birdbrain?",
    "section": "",
    "text": "Langauge models have…",
    "crumbs": [
      "Why Ibis Birdbrain?"
    ]
  }
]