Guardrails Local RAG Bot
Description: Privacy-first AI document assistant secured by NVIDIA NeMo Guardrails.

Document Upload
Drop files here or browse
PDF · TXT · DOCX · DOC
Chat with Your Documents
Upload any PDF, TXT or Word file — your data never leaves your machine. The AI reads only what you share.
1
Upload Your Document
Drop a PDF, TXT or DOCX. It's split into small chunks and converted into vector embeddings stored locally with FAISS.
2
Ask a Question
Your question is embedded and the most relevant chunks are retrieved from the vector store — this is the Retrieval in RAG.
3
Local LLM Answers
The retrieved context + your question go to Ollama running on your machine. No cloud, no data leaks, fully private.
4
Guardrails Filter
Both input and output pass through NeMo Guardrails — blocking jailbreaks, credential leaks and sensitive data exposure.
PDF · TXT · DOCX
Local LLM (Ollama)
NeMo Guardrails
FAISS Vector Cache
100% Private
Documents indexed — ask your first question below.
Processing…
G
🛡 Internal — API keys & credentials protected
Connect Your Local Ollama Online

Since this app is hosted on the internet, it can't directly reach `localhost` on your machine. Use a tunnel to bridge the gap.

Option A — ngrok Free

1. Install: `choco install ngrok` or download from ngrok.com/download
2. Start Ollama: `ollama serve`
3. Run tunnel: `ngrok http 11434 --host-header="localhost:11434"`
4. Copy the `https://xxxx.ngrok-free.app` URL
5. Paste it into the Endpoint URL field in the sidebar

Option B — Cloudflare Tunnel Free · No account needed

1. Start Ollama: `ollama serve`
2. Run: `cloudflared tunnel --url http://localhost:11434`
3. Copy the `https://xxx.trycloudflare.com` URL
4. Paste it into the Endpoint URL field in the sidebar

⚠ Security: Anyone with the tunnel URL can access your Ollama. Close the tunnel (`Ctrl+C`) when not in use.
Start Ollama on Your Computer

Automatic startup failed. Please ensure Ollama is installed on your computer and start it manually using the terminal:

Step 1 — Install Ollama One-time

Download from ollama.com/download and install for your OS.

Step 2 — Start the Ollama server

1. Open Terminal (macOS / Linux) or Command Prompt / PowerShell (Windows)
2. Run: `ollama serve`
3. Leave that window open — Ollama keeps running in the background

Step 3 — Pull a model If you haven't yet

1. In a new terminal window, run: `ollama pull gemma3:1b`
2. Or any other model: `ollama pull llama3.2:1b`

⏳ Auto-detecting… This window will close automatically once Ollama is detected running at `localhost:11434`.