# Running a local MiniLM embedding server with Hugging Face TEI

Only **OpenAI-compatible endpoints** are supported locally.

You can spin up a embedding API based on **sentence-transformers/all-MiniLM-L6-v2** using [Hugging Face TEI](https://github.com/huggingface/text-embeddings-inference):

```bash
docker run --rm -p 8080:80 ghcr.io/huggingface/text-embeddings-inference:cpu-1.8 \
    --model-id sentence-transformers/all-MiniLM-L6-v2
```

---

## Environment variables

Point your backend to the local endpoint and declare the new vector size:

```env
OPENAI_BASE_URL=http://localhost:8080/v1
EMBEDDING_DIMENSION=384
EMBEDDING_MAX_BATCH_SIZE=32 # Not required when using OpenAI embeddings
```

Depending on the model, you might want to change the `EMBEDDING_FALLBACK_MAX_TOKENS` and `EMBEDDING_MAX_BATCH_SIZE` settings, which are set conservatively and according to the requirements of the setup used in this example.

---

## Apply the schema change

With these new settings run:

```bash
dotenv run python main.py embedding resize
```

**Note** that this will delete all records and you will have to re-index.

---

## Re-index embeddings

```bash
dotenv run python main.py index subscriptions
```

The search index now uses **384-dimension MiniLM vectors** served from your local Docker container. That’s it! 🚀
