# vstash End-to-End Test Report

**Date:** 2026-03-23 19:02
**Backend:** cerebras / gpt-oss-120b
**Embedding:** BAAI/bge-small-en-v1.5

**Corpus:** 8 documents, 81 chunks, 3.09 MB

---

## Query 1: "Explain the difference between DARTS and evolutionary NAS methods, including their computational costs"

### ⏱️ Timing

| Phase | Time |
|---|---|
| Embed query | 138.7ms |
| Vector + FTS search | 4.6ms |
| **Total retrieval** | **143.4ms** |
| LLM inference | 0.83s |
| **End-to-end** | **0.97s** |

### 📚 Retrieved Sources

| # | Source | Score |
|---|---|---|
| 1 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 23) | 0.01 |
| 2 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 21) | 0.009836 |
| 3 | Vstash Constitution (chunk 1) | 0.009677 |
| 4 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 10) | 0.009524 |
| 5 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 2) | 0.009375 |

### 💬 Answer (33 words)

The provided excerpts do not contain any information about DARTS, evolutionary Neural Architecture Search (NAS) methods, or their computational costs. Therefore, I cannot explain the difference between them based on the given context.

---

## Query 2: "What are the main arguments for renewable energy, and how do costs compare to fossil fuels?"

### ⏱️ Timing

| Phase | Time |
|---|---|
| Embed query | 11.7ms |
| Vector + FTS search | 3.5ms |
| **Total retrieval** | **15.2ms** |
| LLM inference | 4.33s |
| **End-to-end** | **4.34s** |

### 📚 Retrieved Sources

| # | Source | Score |
|---|---|---|
| 1 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 10) | 0.01 |
| 2 | https://en.wikipedia.org/wiki/Model_context_protocol (chunk 10) | 0.009836 |
| 3 | https://en.wikipedia.org/wiki/Model_context_protocol (chunk 6) | 0.009677 |
| 4 | https://en.wikipedia.org/wiki/Model_context_protocol (chunk 20) | 0.009524 |
| 5 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 20) | 0.009375 |

### 💬 Answer (43 words)

The provided contexts discuss autonomous LLM agents, multi‑plan generation, reflection, and the Model Context Protocol. They do not contain any information about arguments for renewable energy or cost comparisons with fossil fuels. Therefore, I cannot answer that question based on the given material.

---

## Query 3: "Describe Alice's emotional journey and key character interactions in Wonderland"

### ⏱️ Timing

| Phase | Time |
|---|---|
| Embed query | 16.9ms |
| Vector + FTS search | 2.2ms |
| **Total retrieval** | **19.0ms** |
| LLM inference | 1.37s |
| **End-to-end** | **1.39s** |

### 📚 Retrieved Sources

| # | Source | Score |
|---|---|---|
| 1 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 22) | 0.01 |
| 2 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 26) | 0.009836 |
| 3 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 25) | 0.009677 |
| 4 | https://en.wikipedia.org/wiki/Retrieval-augmented_generation (chunk 16) | 0.009524 |
| 5 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 0) | 0.009375 |

### 💬 Answer (39 words)

The provided contexts discuss autonomous large‑language‑model agents, their architectures, surveys, and related research. They contain no information about Alice, her emotional journey, or any character interactions in *Wonderland*. Therefore, I cannot describe those details based on the given material.

---

## Query 4: "What does Sun Tzu say about the importance of knowing your enemy and terrain?"

### ⏱️ Timing

| Phase | Time |
|---|---|
| Embed query | 15.9ms |
| Vector + FTS search | 1.4ms |
| **Total retrieval** | **17.4ms** |
| LLM inference | 1.08s |
| **End-to-end** | **1.1s** |

### 📚 Retrieved Sources

| # | Source | Score |
|---|---|---|
| 1 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 20) | 0.01 |
| 2 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 6) | 0.009836 |
| 3 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 24) | 0.009677 |
| 4 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 1) | 0.009524 |
| 5 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 13) | 0.009375 |

### 💬 Answer (20 words)

The provided contexts do not contain any statement from Sun Tzu about the importance of knowing your enemy and terrain.

---

## Query 5: "How should a production API implement rate limiting and caching with Redis?"

### ⏱️ Timing

| Phase | Time |
|---|---|
| Embed query | 19.6ms |
| Vector + FTS search | 3.2ms |
| **Total retrieval** | **22.8ms** |
| LLM inference | 1.29s |
| **End-to-end** | **1.31s** |

### 📚 Retrieved Sources

| # | Source | Score |
|---|---|---|
| 1 | Vstash Constitution (chunk 2) | 0.01 |
| 2 | Vstash Constitution (chunk 3) | 0.009836 |
| 3 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 18) | 0.009677 |
| 4 | Readme (chunk 1) | 0.009524 |
| 5 | Vstash Constitution (chunk 1) | 0.009375 |

### 💬 Answer (44 words)

The provided documents describe the Vstash tool‑chain (ingestion, embedding, hybrid search, CLI, roadmap, privacy, etc.), but they do not contain any information about implementing rate‑limiting or caching for a production API with Redis. Consequently, I cannot answer that question based on the given context.

---

## Query 6: "What is vstash's technology stack and how does it achieve fast local search?"

### ⏱️ Timing

| Phase | Time |
|---|---|
| Embed query | 11.3ms |
| Vector + FTS search | 1.0ms |
| **Total retrieval** | **12.3ms** |
| LLM inference | 0.97s |
| **End-to-end** | **0.99s** |

### 📚 Retrieved Sources

| # | Source | Score |
|---|---|---|
| 1 | Vstash Test Frontmatter (chunk 0) | 0.01 |
| 2 | Readme (chunk 0) | 0.009836 |
| 3 | Vstash Constitution (chunk 0) | 0.009677 |

### 💬 Answer (368 words)

**vstash’s technology stack**

| Layer | Technology | Why it’s used |
|-------|------------|---------------|
| Vector store | **sqlite‑vec** | Stores embeddings in a single SQLite file; provides fast cosine‑similarity search without any external server. |
| Keyword search | **FTS5 (SQLite)** | Built‑in full‑text search with exact matches and Porter stemming; runs locally inside the same SQLite database. |
| Hybrid ranking | **Reciprocal Rank Fusion (RRF)** | Combines the results of vector search and keyword search to give better relevance without needing extra training. |
| Embeddings | **FastEmbed (ONNX Runtime)** | Generates embeddings locally on‑CPU (~700 chunks / s) using small models like `BAAI/bge‑small‑en‑v1.5`. |
| Embedding model | **BAAI/bge‑small‑en‑v1.5** (384‑dim) | Fastest quality‑/‑speed trade‑off; dimensions match the vector store. |
| Inference back‑ends | **Cerebras API**, **OpenAI API**, **Ollama (local)** | Configurable; Ollama keeps everything local, Cerebras/OpenAI send only the top‑k chunks. |
| Document parsing | **markitdown** | Parses PDFs, DOCX, PPTX, HTML, Markdown, code files, and URLs locally. |
| Configuration | **Pydantic v2** | Type‑safe, validated `vstash.toml`. |
| CLI | **Typer + Rich** | Provides a clean, interactive terminal UI. |
| Language | **Python 3.10+** | Easy ecosystem and rapid development. |

**How fast local search is achieved**

1. **All data lives in one SQLite `.db` file** – document metadata, chunk text, embeddings, and FTS5 indexes are stored together, eliminating network latency and the overhead of multiple services. *(Context 3, “Technology Stack”)*  
2. **Vector similarity is done with `sqlite‑vec`** – cosine‑similarity queries run directly inside SQLite, giving sub‑second retrieval even for millions of vectors. *(Context 3, “Technology Stack”)*  
3. **Keyword matching uses SQLite’s built‑in FTS5** – no external search engine is required; exact‑match and stemming queries are executed instantly. *(Context 3, “Technology Stack”)*  
4. **Hybrid ranking via Reciprocal Rank Fusion** merges the semantic (vector) and lexical (FTS5) results, improving relevance without extra model inference. *(Context 1, “Hybrid Search” and Context 3, “Technology Stack”)*  
5. **Embeddings are generated locally with FastEmbed (ONNX)** – ~700 chunks per second on CPU, so indexing new documents is quick and stays on‑device. *(Context 2, “Embeddings” table and Context 3, “Embeddings” entry)*  

Together, these components provide **instant, sub‑second semantic search** while keeping everything local and server‑less.

---

## Query 7: "How does Retrieval-Augmented Generation reduce hallucinations in language models?"

### ⏱️ Timing

| Phase | Time |
|---|---|
| Embed query | 10.6ms |
| Vector + FTS search | 2.9ms |
| **Total retrieval** | **13.6ms** |
| LLM inference | 0.74s |
| **End-to-end** | **0.75s** |

### 📚 Retrieved Sources

| # | Source | Score |
|---|---|---|
| 1 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 21) | 0.01 |
| 2 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 23) | 0.009836 |
| 3 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 26) | 0.009677 |
| 4 | https://en.wikipedia.org/wiki/Retrieval-augmented_generation (chunk 10) | 0.009524 |

### 💬 Answer (29 words)

The provided excerpts do not contain an explanation of how Retrieval‑Augmented Generation (RAG) reduces hallucinations in language models. Therefore, I cannot answer the question based on the given context.

---

## Summary

| Metric | Value |
|---|---|
| Queries tested | 7 |
| Total retrieval time | 244ms (35ms avg) |
| Total LLM time | 10.6s (1.52s avg) |
| Avg end-to-end | 1.55s |
| Corpus size | 81 chunks |