# vstash End-to-End Test Report

**Date:** 2026-03-23 20:24
**Backend:** cerebras / gpt-oss-120b
**Embedding:** BAAI/bge-small-en-v1.5

**Corpus:** 8 documents, 81 chunks, 3.09 MB

---

## Query 1: "Explain the difference between DARTS and evolutionary NAS methods, including their computational costs"

### ⏱️ Timing

| Phase | Time |
|---|---|
| Embed query | 7.9ms |
| Vector + FTS search | 3.4ms |
| **Total retrieval** | **11.4ms** |
| LLM inference | 0.77s |
| **End-to-end** | **0.79s** |

### 📚 Retrieved Sources

| # | Source | Score |
|---|---|---|
| 1 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 23) | 0.01 |
| 2 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 21) | 0.009836 |
| 3 | Vstash Constitution (chunk 1) | 0.009677 |
| 4 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 10) | 0.009524 |
| 5 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 2) | 0.009375 |

### 💬 Answer (33 words)

The provided excerpts do not contain any information about DARTS, evolutionary neural architecture search (NAS) methods, or their computational costs. Therefore, I cannot explain the difference between them based on the given context.

---

## Query 2: "What are the main arguments for renewable energy, and how do costs compare to fossil fuels?"

### ⏱️ Timing

| Phase | Time |
|---|---|
| Embed query | 9.5ms |
| Vector + FTS search | 2.1ms |
| **Total retrieval** | **11.6ms** |
| LLM inference | 0.96s |
| **End-to-end** | **0.98s** |

### 📚 Retrieved Sources

| # | Source | Score |
|---|---|---|
| 1 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 10) | 0.01 |
| 2 | https://en.wikipedia.org/wiki/Model_context_protocol (chunk 10) | 0.009836 |
| 3 | https://en.wikipedia.org/wiki/Model_context_protocol (chunk 6) | 0.009677 |
| 4 | https://en.wikipedia.org/wiki/Model_context_protocol (chunk 20) | 0.009524 |
| 5 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 20) | 0.009375 |

### 💬 Answer (43 words)

The provided contexts discuss autonomous LLM agents, multi‑plan generation, reflection, and the Model Context Protocol. They do not contain any information about arguments for renewable energy or cost comparisons with fossil fuels. Therefore, I cannot answer that question based on the given material.

---

## Query 3: "Describe Alice's emotional journey and key character interactions in Wonderland"

### ⏱️ Timing

| Phase | Time |
|---|---|
| Embed query | 11.0ms |
| Vector + FTS search | 2.5ms |
| **Total retrieval** | **13.5ms** |
| LLM inference | 1.24s |
| **End-to-end** | **1.25s** |

### 📚 Retrieved Sources

| # | Source | Score |
|---|---|---|
| 1 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 22) | 0.01 |
| 2 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 26) | 0.009836 |
| 3 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 25) | 0.009677 |
| 4 | https://en.wikipedia.org/wiki/Retrieval-augmented_generation (chunk 16) | 0.009524 |
| 5 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 0) | 0.009375 |

### 💬 Answer (39 words)

The provided contexts discuss autonomous large‑language‑model agents, their architectures, surveys, and related research. They contain no information about Alice, her emotional journey, or any character interactions in *Wonderland*. Therefore, I cannot describe those details based on the given material.

---

## Query 4: "What does Sun Tzu say about the importance of knowing your enemy and terrain?"

### ⏱️ Timing

| Phase | Time |
|---|---|
| Embed query | 15.9ms |
| Vector + FTS search | 1.3ms |
| **Total retrieval** | **17.2ms** |
| LLM inference | 0.81s |
| **End-to-end** | **0.83s** |

### 📚 Retrieved Sources

| # | Source | Score |
|---|---|---|
| 1 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 20) | 0.01 |
| 2 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 6) | 0.009836 |
| 3 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 24) | 0.009677 |
| 4 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 1) | 0.009524 |
| 5 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 13) | 0.009375 |

### 💬 Answer (20 words)

The provided contexts do not contain any statement from Sun Tzu about the importance of knowing your enemy and terrain.

---

## Query 5: "How should a production API implement rate limiting and caching with Redis?"

### ⏱️ Timing

| Phase | Time |
|---|---|
| Embed query | 10.3ms |
| Vector + FTS search | 1.4ms |
| **Total retrieval** | **11.7ms** |
| LLM inference | 0.76s |
| **End-to-end** | **0.77s** |

### 📚 Retrieved Sources

| # | Source | Score |
|---|---|---|
| 1 | Vstash Constitution (chunk 2) | 0.01 |
| 2 | Vstash Constitution (chunk 3) | 0.009836 |
| 3 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 18) | 0.009677 |
| 4 | Readme (chunk 1) | 0.009524 |
| 5 | Vstash Constitution (chunk 1) | 0.009375 |

### 💬 Answer (48 words)

The provided documents describe the vstash project’s architecture, ingestion and query pipelines, chunking strategy, configuration, roadmap, and privacy model. They do not contain any information about implementing rate limiting or caching for a production API with Redis. Therefore, I cannot answer that question based on the given context.

---

## Query 6: "What is vstash's technology stack and how does it achieve fast local search?"

### ⏱️ Timing

| Phase | Time |
|---|---|
| Embed query | 14.3ms |
| Vector + FTS search | 1.5ms |
| **Total retrieval** | **15.8ms** |
| LLM inference | 1.89s |
| **End-to-end** | **1.9s** |

### 📚 Retrieved Sources

| # | Source | Score |
|---|---|---|
| 1 | Vstash Test Frontmatter (chunk 0) | 0.01 |
| 2 | Readme (chunk 0) | 0.009836 |
| 3 | Vstash Constitution (chunk 0) | 0.009677 |

### 💬 Answer (387 words)

**vstash’s technology stack**

| Layer | Technology | Why it’s used |
|-------|------------|---------------|
| Vector store | **sqlite‑vec** | Stores embeddings in a single SQLite file; enables fast cosine‑similarity search without a separate server. |
| Keyword search | **FTS5 (SQLite)** | Built‑in full‑text index with exact‑match and Porter stemming; runs locally in the same DB. |
| Hybrid ranking | **Reciprocal Rank Fusion (RRF)** | Combines the semantic scores from sqlite‑vec with the keyword scores from FTS5 to give better results without any model training. |
| Embeddings | **FastEmbed (ONNX Runtime)** | Generates embeddings locally on CPU (~700 chunks / s) using small models like `BAAI/bge‑small‑en‑v1.5`. |
| Embedding model | **BAAI/bge‑small‑en‑v1.5** (384‑dim) | Fastest quality‑/‑speed ratio; dimensions match the vector store. |
| Inference back‑ends | **Cerebras API**, **OpenAI API**, **Ollama (local)** | Configurable; only the top‑k retrieved chunks are sent to a remote API, or everything stays local with Ollama. |
| Document parsing | **markitdown** | Handles PDF, DOCX, PPTX, HTML, Markdown, code files, and URLs locally. |
| Storage | **SQLite (single .db file)** | All metadata, chunk text, embeddings, and FTS5 indexes live in one file that can be copied or backed up. |
| Configuration | **Pydantic v2** | Type‑safe config handling. |
| CLI | **Typer + Rich** | Provides a clean, interactive terminal UI. |
| Language | **Python 3.10+** | Easy ecosystem and rapid development. |

**How fast local search is achieved**

1. **All data lives in one SQLite database** – no external server, no network latency.  
2. **Vector similarity** is performed by `sqlite‑vec`, which computes cosine similarity directly inside SQLite, giving sub‑second semantic retrieval.  
3. **Keyword matching** uses SQLite’s built‑in `FTS5` index, which is highly optimized for fast exact‑match and stemming searches.  
4. **Hybrid ranking with RRF** merges the two result sets efficiently, improving relevance without extra computation.  
5. **Embeddings are generated locally** with FastEmbed (ONNX), avoiding remote calls and allowing ~700 chunks per second.  
6. **Chunking** creates overlapping token windows (default 1024‑token size, 128‑token overlap) that keep retrieval fast and accurate.  

Together, these components let vstash return semantic answers in under a second, entirely on the user’s machine.  

*Sources: Context 3 “Technology Stack” table and description of fast local components; Context 1 “Hybrid Search” and “Storage Layer”; Context 2 “Why vstash?” and “Embedding” sections.*

---

## Query 7: "How does Retrieval-Augmented Generation reduce hallucinations in language models?"

### ⏱️ Timing

| Phase | Time |
|---|---|
| Embed query | 24.2ms |
| Vector + FTS search | 2.0ms |
| **Total retrieval** | **26.2ms** |
| LLM inference | 0.63s |
| **End-to-end** | **0.66s** |

### 📚 Retrieved Sources

| # | Source | Score |
|---|---|---|
| 1 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 21) | 0.01 |
| 2 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 23) | 0.009836 |
| 3 | The Architecture And Evolution Of Autonomous Llm Agents   Notebooklm (chunk 26) | 0.009677 |
| 4 | https://en.wikipedia.org/wiki/Retrieval-augmented_generation (chunk 10) | 0.009524 |

### 💬 Answer (17 words)

The provided excerpts do not contain an explanation of how Retrieval‑Augmented Generation reduces hallucinations in language models.

---

## Summary

| Metric | Value |
|---|---|
| Queries tested | 7 |
| Total retrieval time | 107ms (15ms avg) |
| Total LLM time | 7.1s (1.01s avg) |
| Avg end-to-end | 1.02s |
| Corpus size | 81 chunks |