---
name: docs-rag-vectors
description: Generate embeddings and SQL for syncing documentation to PostgreSQL vector database
---

# Documentation Embeddings Workflow

**Goal:** Generate embeddings for documentation chunks and create a SQL script for importing to PostgreSQL.

**This workflow is now implemented as a CLI command.** Run:

```bash
aidocs rag-vectors
```

---

## CLI COMMAND USAGE

```bash
aidocs rag-vectors                    # Generate sync SQL (smart)
aidocs rag-vectors --dry              # Preview what would be synced
aidocs rag-vectors --force            # Re-sync all (ignore last-sync)
aidocs rag-vectors --table my_docs    # Custom table name
```

---

## PREREQUISITES

1. **Chunks must exist:** Run `aidocs rag-chunks` first
2. **OpenAI API key:** Set `OPENAI_API_KEY` environment variable

---

## WHAT IT DOES

1. Reads `docs/.chunks/manifest.json` for chunk file locations
2. Compares against `docs/.chunks/last-sync.json` to find changes
3. Generates embeddings via OpenAI API (only for new/changed chunks)
4. Creates `docs/.chunks/sync.sql` with INSERT statements
5. Updates `docs/.chunks/last-sync.json` with synced file hashes

---

## SMART SYNC

The command is incremental:

- **Unchanged files** → Skip (no API calls)
- **Changed files** → Re-generate embeddings
- **New files** → Generate embeddings
- **Deleted files** → Add DELETE statements

---

## OUTPUT

```
✅ Embeddings generated!

Files synced: 3
Files deleted: 0
Embeddings: 10
Tokens used: ~4,832

SQL file: docs/.chunks/sync.sql

Import to database:
  psql $DATABASE_URL -f docs/.chunks/sync.sql
```

---

## SQL FORMAT

The generated SQL uses transactions:

```sql
-- Documentation Sync SQL
-- Generated by aidocs rag-vectors at 2024-01-15T11:00:00Z

BEGIN;

-- Delete removed/changed file chunks
DELETE FROM doc_embeddings WHERE file_path = 'docs/old-page.md';

-- Insert new chunks
INSERT INTO doc_embeddings (file_path, content, chunk_index, title, metadata, embedding)
VALUES (
  'docs/campaigns/lifecycle.md',
  'Content here...',
  0,
  'Overview',
  '{"hierarchy": ["Campaigns", "Lifecycle"]}'::jsonb,
  '[0.0023, -0.0145, ...]'::vector
);

COMMIT;
```

---

## EMBEDDING MODEL

Using `text-embedding-3-small`:
- 1536 dimensions
- ~$0.02 per 1M tokens
- Good balance of quality/cost

---

## ERROR HANDLING

| Error | Action |
|-------|--------|
| No manifest | Prompt to run `aidocs rag-chunks` first |
| No API key | Show how to set OPENAI_API_KEY |
| API rate limit | Automatic retry with backoff |
| API error | Show error, suggest checking key/quota |
