# reclm


<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->

When building AI based tooling and packaging we often call LLMs while
prototyping and testing our code. A single LLM call can take 100’s of ms
to run and the output isn’t deterministic. This can really slow down
development especially if our notebook contains many LLM calls 😞.

While LLMs are new, working with external APIs in our code isn’t. Plenty
of tooling already exists that make working with APIs much easier. For
example, Python’s unittest mock object is commonly used to simulate or
mock an API call so that it returns a hardcoded response. This works
really well in the traditional Python development workflow and can make
our tests fast and predictable.

However, it doesn’t work well in the nbdev workflow where oftentimes
we’ll want to quickly run all cells in our notebook while we’re
developing our code. While we can use mocks in our test cells we don’t
want our exported code cells to be mocked. This leaves us with two
choices:

- we temporarily mock our exported code cells but undo the mocking
  before we export these cells.
- we do nothing and just live with notebooks that take a long time to
  run.

Both options are pretty terrible as they pull us out of our flow state
and slow down development 😞.

`reclm` builds on the underlying idea of mocks but adapts them to
exploratory workflows. It initializes each sdk client
(e.g. `AsyncAnthropic`) with a custom `http_client`. This `http_client`
intercepts each LLM call, caches the response, and returns this cached
response if the user makes the same LLM call again.

## Usage

To use `reclm`

- install the package:
  `pip install git+https://github.com/AnswerDotAI/reclm.git`
- import the package `from reclm.core import enable_reclm` in each
  notebook
- add
  [`enable_reclm()`](https://AnswerDotAI.github.io/reclm/core.html#enable_reclm)
  to the top of each notebook

*Note:
[`enable_reclm`](https://AnswerDotAI.github.io/reclm/core.html#enable_reclm)
should be added after you import the OpenAI and/or Anthropic SDK.*

Every LLM call you make using OpenAI/Anthropic will now be cached in
`reclm.json`.

> [!NOTE]
>
> ### Cache Location
>
> If you’re using `reclm` in an nbdev project the cache
> (i.e. `reclm.json`) will be added to your project’s root dir. For all
> other projects the cache will be added to the current working
> directory. You can set a custom location for the cache by passing
> `cache_dir` to
> [`enable_reclm`](https://AnswerDotAI.github.io/reclm/core.html#enable_reclm)
> (e.g. `enable_reclm(cache_dir='/path/to/cache')`).

### Your Tests

`nbdev_test` will automatically read from the cache. However, if your
notebooks contain LLM calls that haven’t been cached, `nbdev_test` will
call the OpenAI/Anthropic APIs and then cache the responses.

### Cleaning the cache

It is recommended that you clean the cache before committing it. This
will remove any stale LLM requests you’ve accumulated during
development.

To clean the cache, run `update_reclm_cache` from your project’s root
directory.

*Note: Your LLM request/response data is stored in a file called
`reclm.json`. Depending on your setup, it might be stored in your
project’s root dir or your current working directory.*

*Note: All request headers are removed so it is safe to include this
file in your version control system (e.g. git). In fact, it is expected
that you’ll include this file in your vcs. There is one scenario where
your api key could end up in the cache. This occurs if you make an LLM
call that includes your api key in the response such as an
authentication error.*

### Reclm Tests

The reclm test suite is found in `test_reclm.py`. Use the command
`python test_reclm.py` to run the test suite.

Each test checks a specific SDK endpoint (e.g. anthropic streaming). If
reclm is working correctly the SDK endpoint won’t be called and instead
the response will be pulled from `reclm.json`.
