{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using Pixi for reproducible environments\n",
    "\n",
    "OpenMDAO requires a number of dependencies to fully take advantage of its capabilities. Sometimes finding versions of dependencies which are compatible and are known to function correctly is difficult.\n",
    "\n",
    "OpenMDAO uses [Pixi](https://pixi.sh) for dependency management. Pixi is especially useful because some OpenMDAO dependencies (like MPI, PETSc) are not available on PyPI but are needed for parallel computing features.\n",
    "\n",
    "Using Pixi allows users and developers to get consistent, reproducible environments.\n",
    "\n",
    "## Installing Pixi\n",
    "\n",
    "Before users can utilize Pixi, it needs to be installed:\n",
    "\n",
    "```bash\n",
    "curl -fsSL https://pixi.sh/install.sh | bash\n",
    "```\n",
    "\n",
    "## Getting the Pixi environments\n",
    "\n",
    "Pixi stores information about OpenMDAO environments in a _manifest_ file (`pixi.toml`) and a _lock_ file (`pixi.lock`) in the root directory of the OpenMDAO git repository.\n",
    "\n",
    "The intent is that the lock file will be tested in our CI machinery, and should provide a consistent environment for as many of our users as possible."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Option 1: Using the environment from the OpenMDAO git repo\n",
    "\n",
    "Users who want to develop OpenMDAO itself can setup the Pixi environments from the cloned repository.\n",
    "\n",
    "```bash\n",
    "# Clone the repository\n",
    "git clone https://github.com/OpenMDAO/OpenMDAO.git\n",
    "\n",
    "# Enter the directory with the pixi.toml and pixi.lock files\n",
    "cd OpenMDAO\n",
    "```\n",
    "\n",
    "### Option 2: Downloading the Pixi manifest and lock files\n",
    "\n",
    "Other users who just want to pip install without cloning the repo will need to dowload the manifest and lock files.\n",
    "These should be placed in the top-level of the users desired working directory/project directory.\n",
    "\n",
    "```bash\n",
    "# Download the latest Pixi configuration...\n",
    "curl -O https://raw.githubusercontent.com/OpenMDAO/OpenMDAO/master/pixi.toml\n",
    "curl -O https://raw.githubusercontent.com/OpenMDAO/OpenMDAO/master/pixi.lock\n",
    "\n",
    "# Alternatively, for a specific release...\n",
    "curl -O https://raw.githubusercontent.com/OpenMDAO/OpenMDAO/3.42.0/pixi.toml\n",
    "curl -O https://raw.githubusercontent.com/OpenMDAO/OpenMDAO/3.42.0/pixi.lock\n",
    "```\n",
    "\n",
    "Pixi files are only available as of OpenMDAO version 3.42.0."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Installing the desired environment\n",
    "\n",
    "From the directory with the manifest and lock files:\n",
    "\n",
    "```bash\n",
    "# Install the default environment (no mpi)\n",
    "pixi install --frozen\n",
    "\n",
    "# Or, install the python 3.13 environment with all dependencies\n",
    "pixi install -e py313 --frozen\n",
    "```\n",
    "\n",
    "Using **--frozen** causes Pixi to use the exact versions for the environment as defined in the `pixi.lock` file. This is what we test against on CI and should be the most reliable environment.\n",
    "\n",
    "You can remove `--frozen` and Pixi will attempt to install the latest possible dependencies for most environments.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using the given environment via `pixi shell`\n",
    "\n",
    "You can then either shell into the environment (more akin to activating a conda environment), or you can run a command using that environment.\n",
    "\n",
    "```bash\n",
    "# Shell into the py313 environment (if installed)...\n",
    "pixi shell -e py313\n",
    "\n",
    "# Install OpenMDAO in developer mode (if in the OpenMDAO git repo)...\n",
    "python -m pip install -e .\n",
    "\n",
    "# OR, Install OpenMDAO from pypi...\n",
    "python -m pip install openmdao\n",
    "```\n",
    "\n",
    "That last step may seem a bit odd, but OpenMDAO is not a dependency of itself. **You will need to install OpenMDAO into whatever pixi environment you are using.** You only need to do this once after a given environment is installed.\n",
    "\n",
    "Now the environment is ready to be used\n",
    "\n",
    "```bash\n",
    "# Run a script in the environment that we have shelled into...\n",
    "python my_openmdao_script.py\n",
    "```\n",
    "\n",
    "More information about our Pixi implementation, including available environments and other ways of using it, are avaiable in the developer section of our documentation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using the given environment via `pixi run`\n",
    "\n",
    "Alternatively, users can run commands given a desired Pixi environment without shelling into it.\n",
    "\n",
    "As before, make sure OpenMDAO has been installed in the environment:\n",
    "\n",
    "```bash\n",
    "# Install OpenMDAO from pypi into the all environment\n",
    "pixi run -e py313 python -m pip install openmdao\n",
    "```\n",
    "\n",
    "Or, install a local OpenMDAO git repository in developer mode into the all environment:\n",
    "```bash\n",
    "# Install OpenMDAO from a local git repository\n",
    "cd OpenMDAO\n",
    "pixi run python -m pip install -e .\n",
    "```\n",
    "\n",
    "And now, run an OpenMDAO python script using the given environment\n",
    "```bash\n",
    "pixi run python path/to/my_openmdao_script.py\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Available Environments\n",
    "\n",
    "### **default**\n",
    "\n",
    "General development environment with JAX, numba, visualization, notebooks, and testing support, but no MPI. Uses the latest solvable environment.\n",
    "\n",
    "Best for most users and everyday development work.\n",
    "Usage: pixi shell  (or pixi shell -e default)\n",
    "\n",
    "### **minimal**\n",
    "\n",
    "Default dependencies only plus testing related dependencies, with latest solvable environment.\n",
    "\n",
    "Used for testing in the absence of dependencies.\n",
    "Usage: `pixi shell -e minimal`\n",
    "\n",
    "### **py311, py312, py313**\n",
    "\n",
    "Comprehensive environment with all optional dependencies except documenation and testing-related ones. Pinned to a specific python version.\n",
    "\n",
    "This will give most users the \"full\" OpenMDAO capability.\n",
    "Usage:`pixi shell -e py313`\n",
    "\n",
    "### **dev**\n",
    "\n",
    "The latest solvable environment with the addition of documentation and packaging-related dependencies.\n",
    "\n",
    "Usage: `pixi shell -e docs`\n",
    "\n",
    "### **oldest**\n",
    "\n",
    "Includes oldest supported versions of all dependencies except packaging-related ones.\n",
    "\n",
    "Usage: `pixi shell -e oldest`\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What if the latest solvable environment has known bugs?\n",
    "\n",
    "If a dependency is released with a change that breaks our functionality due to a bug, we will pin that dependency to the latest known working version on our master repository (and in any releases) until it is fixed.\n",
    "\n",
    "Reach out to the team with an issue or discussion on our [Github repostiory](https://github.com/OpenMDAO/OpenMDAO)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using OpenMDAO's environments for Github actions in dependent projects\n",
    "\n",
    "OpenMDAO's repository contains an action definition that can be used for Github actions by other projects.  This might be convenient for dependent projects to streamline their CI process and ensure they're using the same repos that we've tested against.\n",
    "\n",
    "Adding the following step to your workflow/action will install the desired OpenMDAO pixi environement with the given version of OpenMDAO.\n",
    "\n",
    "```\n",
    "      - name: 'Setup OpenMDAO pixi Environment'\n",
    "        id: setup_pixi_environment\n",
    "        uses: OpenMDAO/OpenMDAO/.github/actions/setup_openmdao_pixi@master\n",
    "        with:\n",
    "          environment: 'py312'\n",
    "          openmdao_pixi_ref: 'master'\n",
    "          openmdao_install_from: 'pypi'\n",
    "          openmdao_version: ''\n",
    "```\n",
    "\n",
    "When called from another repo, this will download the OpenMDAO `pixi.toml` and `pixi.lock` files from the given git ref and save them in `${{ github.workspace }}/.openmdao-pixi/`.\n",
    "It will also export the environment variable `$PIXI_MANIFEST` to the available GitHub environment variables.\n",
    "\n",
    "Then, in subsequent steps, you can use this environment by activating the environment in a bash-running step, for instance:\n",
    "\n",
    "```\n",
    "   - name: Run tests\n",
    "        id: run_tests\n",
    "        run: |\n",
    "          eval \"$(pixi shell-hook -e py313 --manifest-path=${{ env.PIXI_MANIFEST }})\"\n",
    "          testflo my_repo -n 4\n",
    "```        \n",
    "\n",
    "Options:\n",
    "\n",
    "The `with:` section allows you to specify the following arguments:\n",
    "\n",
    "- **environment**\n",
    "  \n",
    "  Pixi environment(s) to install\n",
    "  \n",
    "  - Required: false\n",
    "  - Default: `'py313'`\n",
    "\n",
    "- **frozen**\n",
    "  \n",
    "  Use frozen lock file (recommended for CI). Setting this to false will attempt to find newer versions of dependencies.\n",
    "  \n",
    "  - Required: false\n",
    "  - Default: `'true'`\n",
    "\n",
    "- **openmdao_pixi_ref**\n",
    "\n",
    "  The git reference from which the OpenMDAO pixi files should be installed, when called from another repo.\n",
    "  \n",
    "  - Required: false\n",
    "  - Default: `'master'`\n",
    "\n",
    "- **openmdao_install_from**\n",
    "  \n",
    "  How to install OpenMDAO: pypi, github, wheel, local, local_developer, or none.\n",
    "  Generally, other repositories should only be using the pypi or github options here, as the others assume the action is operating on the OpenMDAO repo itself.\n",
    "  \n",
    "  - Required: false\n",
    "  - Default: `'pypi'`\n",
    "\n",
    "- **openmdao_version**\n",
    "  \n",
    "  OpenMDAO version (for PyPI and Github installs only): \"\" (latest) or str such as \"3.42.0\".\n",
    "  \n",
    "  - Required: false\n",
    "  - Default: `''`\n"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Tags",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
