# MCP: *M*odeling *C*ollaborators *P*rogrammatically

**MCP** aims to investigate how humans model collaborators using different context-appropriate "function calls" which estimate agent properties (e.g., effort/competence) and group states with varying computational cost.

---

This repo should contain core project-specific code in the future (e.g., my new version of the task, and my analysis), following the lab's template repo. For now, this repo serves as an entry point for the MCP project, which is hosted in personal repos for practical purposes (including to enable github pages for free):

- Replication project: https://github.com/psyc-201/xiang2023
- Memo programming sandbox: https://github.com/jczimm/memo-sandbox
- Writing: https://github.com/jczimm/fyp-writing

---

<!-- # Python project template -->

This is a ready-to-go project template for setting up a new Python project with the lab's recommended folder organization structure and project management-tools. It's designed to help you bootstrap a new Python project quickly using [Pixi](https://pixi.sh/latest/) to manage libraries and common tasks. 

> **Important Prerequisites**  
> *You might already be good-to-go if you previously followed the "Setting up your knowledge worker's toolbox" page on the lab wiki*  
> 1. Check if you have Homebrew installed `which brew` and follow [these instructions](https://brew.sh/) if not
> 2. Check if you have Pixi installed (`which pixi`) and use `brew install pixi` if not

## Instructions

1. Click the green "Use this template" button on the top right and choose a new repository to make
2. Open a terminal and `git clone` that repository to your computer
3. `cd project-name/` move into the folder
4. Run `pixi install` to set up all the packages (will create a hidden .pixi/ folder)
4. Run `pixi run setup` to setup Python
5. Open the project in VSCode/Cursor
6. Install recommended extensions (you should see some pop-ups prompting you)
7. Edit `pyproject.toml` to change the project's author name and email
8. Read through this whole README.md
9. Check out any of the files in `code/` for more details

## Overview 

This template will setup the following:
- A standard set of organized project folders
- An isolated Python environment with some common analysis packages pre-installed (e.g. [`pymer4`](https://eshinjolly.com/pymer4/) for linear-mixed-modeling)
- Integration with your code-editor (VSCode/Cursor) to work *interactively* with Python using Jupyter, Quarto, or Marimo notebooks
- Automatic code-formatting and error-checking as you save your work
- Additional code-editor (VSCode/Cursor) extensions for working with Python, Git, spreadsheets/csv files, and PDFs

The basic project structure made for you will look like this:

```
analysis/          # (empty) to store files you create from processing anything in data/
code/              
    notebooks/     # Jupyter/Quarto/Marimo noteboooks (`.ipynb`, `.qmd`, `.py`) files
    scripts/       # Python scripts (`.py`) files
data/              # (empty) to store raw data files (e.g. `.csv`)
docs/              # (empty) to store additional documents, links to manuscripts, etc
figures/           # (empty) to store generated from notebooks and scripts
src/
   lib/            # Python code you want to reuse across any files in code/    
```

## Managing Python Libraries

You'll typically want to use additional libraries based on your project needs (e.g. fMRI, facial expression analysis). To make things easy, all project configuration and libraries are tracked in `pyproject.toml`. You typically **do not** need to edit this file manually. Instead you should use the Pixi commands below to update this file for you.

### Adding libraries
To add new packages just use:

`pixi add packagename`

which will automatically add a new entry to `pyproject.toml` for you.

If the package is not found it might only be available on pypi, in which case you can use:  

`pixi add --pypi packagename`

You can also install packages from github, for example to try out another open-source library:

`pixi add --pypi "packagename@git+https://github.com/author/packagename.git"`

And even install from a specific *branch*

`pixi add --pypi "packagename@git+https://github.com/author/packagename.git@branch"`

### Removing libraries
You can remove packages just as easily:

`pixi remove packagename`

Or if you used pypi/github:  

`pixi remove --pypi packagename`

### Listing Installed Libraries
You can see the list of the project dependencies you added just by look at the `pyproject.toml` file. But pixi can also show you *all* the libraries in this project using:

`pixi list`

This includes everything in `pyproject.toml` *plus* everything those libraries depend on too.

### Making Pixi aware of manual updates to `pyproject.toml`

Sometimes you might need to edit `pyproject.toml` by hand. In these cases just run `pixi install` after you're done and Pixi will make sure everything is up-to-date.

### Resetting Python
If something ever breaks you can always do a *fres reinstall* of all your libraries:

```
rm .pixi      # Remove all files installed by pixi
rm pixi.lock  # Remove the internal ledger Pixi keeps of all packages
pixi install  # Reinstall everything based on pyproject.toml
pixi setup    # Re-setup Python for interactive data analysis

```

## Best Practices

### Commit your work *often*

The more frequently you `git commit` the easier it is to jump back to a specific change that you made. There's also more advanced features like `git branch` that let you manage different sets of changes in parallel. Don't be afraid to try things out because it's very hard to mess things up if you `git commit` frequently! This is exactly what `git` is for!

It's also a good idea to keep a separate *human-readable* log of your project's progress. You can manage this file with git if you prefer (e.g. using a markdown file) but it's often nicer to have separate log in your notes/todo/project-management app of choice. That way you can refer to your notes and progress more easily.

#### Tip: Working with large files

You should avoid using `git add` to keep track of large files (e.g. videos, big PDFs, fMRI data, anything more than a few mb). `git` will complain and eventually refuse as it's designed to work primarily with text-base files. Instead tell git to *ignore* these files by adding new lines to `.gitignore`. 

You can ignore specific files by name, folder, or file extension, e.g `.pdf` would ignore all PDFs. Ask the lab if you need another solution for managing large files.

### Perform data-analysis *interactively*

This template sets up a few ways for you to perform data analyses interactively:

You can use a [Python Notebook](https://code.visualstudio.com/docs/datascience/jupyter-notebooks) which is a single file that contains code, prose, and figures all in once place and is great for executing code interactively. Just create a new `.ipynb` file in `code/notebooks/` and select the kernel based on the instructions in `code/notebooks/notebook-demo.ipynb`.

Alternatively you can create a a regular Python file (`.py`) and use code-comments to make [code cells](https://code.visualstudio.com/docs/python/jupyter-support-py#_jupyter-code-cells) that you can run with the [interactive window](https://code.visualstudio.com/docs/python/jupyter-support-py). This allows you to work with each part of your file (code, output, plots, dataframes) as separate panels and will feel familiar coming from RStudio/Matlab. 

Another approach is using [Quarto](https://quarto.org/docs/computations/python.html) documents by creating a new `.qmd` file in `code/notebooks/`. Quarto mixes the best of Python scripts + notebooks and can preview a file by running "Quarto Preview" from the command palette (`cmd+shift+p`). Alternateively, we've setup a [pixi task](#running-custom-commands-with-pixi-tasks) you can run to preview any file you create that will auto-update as you work: `pixi run preview quarto-demo.qmd`

Finally you can try [Marimo](https://docs.marimo.io/guides/) which is an even more modern take on notebooks that might be easier to get started with. It also has some neat features for building simple interactive interfaces (e.g. buttons, sldies).

### Use a mini-library to store functions you use a lot

It's very helpful to create common functions and tools that you use over-and-over again across multiple notebooks and files in `code/` (e.g. `load_participant()`). Rather that recreating this each time in every file, put the function in a new file in `src/mcp_hmm/` and it will be importable by any file in `code/` making it much more reusable. In the future you can also easily publish this as a full shareable Python library!

**N.B. This can contain marimo notebooks! So use `notebooks/` for rougher work and `src/mcp_hmm/` to "export" functions for other projects.**

### Make a script to automate repetitive tasks

Another common workflow is automating some kind of task you want to do over and over again. For example, pre-processing data from multiple participants using different data files. For these situations careate a regular Python file (`.py`) in `code/scripts` and run it with `pixi run python code/scripts/yourscript.py participantID` 

You can create a [pixi task](#running-custom-commands-with-pixi-tasks) if typing that all out gets annoying

### Working with Python from the Terminal

If you need work with Python or any of the libraries from your terminal you need to *activate* your environment first. This is helpful if you are following any guides or tutorials online that recommend using command `conda activate` or `source venv/bin/activate`. 

Instead use: `pixi shell`

Now if you run `which python` it points to the Python (and libraries) managed by Pixi.  
You can use `ctrl+d` or type `exit` to *deactivate* the environment

### Running custom commands with Pixi tasks

Pixi has a neat ability to create custom *tasks*. These are custom commands you using with `pixi run mytask`. Tasks are any commands you would normally type in your terminal. This template includes a demo task you try with:

`pixi run demo`

If you inspect `pyproject.toml` you'll see exactly what the `demo` command does in the `pixi.tasks` section. You can add new tasks by adding them to this file or by using the `pixi task add taskname taskdefinition`. We created `demo` using `pixi task add demo python code/scripts/script-demo.py`

Tasks can accept arguments which makes then handy for making Python scripts. Try this replacing `yourname` with your actual name and `yourage` in years (rounded to the nearest year):

`pixi run demo yourname yourage`

You can imagine how this can be handy for creating scripts to automate things like data preprocessing, cleaning, plotting, etc.


