# ThriftLM

Stop paying for the same LLM call twice just because users phrased it differently.

ThriftLM is a drop-in semantic caching layer for LLM applications. It embeds each query with SBERT, checks a vector store for semantically similar past queries, and returns the cached response when similarity is above threshold — no LLM call required.

## Install

```bash
pip install thriftlm
```

## Quickstart

```python
from thriftlm import SemanticCache

cache = SemanticCache(api_key="sc_xxx")
response = cache.get_or_call(query, llm_fn=my_langchain_chain.invoke)
```

That's it. No architecture changes — just wrap the existing LLM call.

## Self-hosting

```bash
cp .env.example .env  # fill in SUPABASE_URL, SUPABASE_KEY, REDIS_URL
docker compose up
```

Set `THRIFTLM_URL=http://localhost:8000` in your app to point at the local instance.

## Docs

[thriftlm.dev/docs](https://thriftlm.dev/docs)

## License

MIT
