# index1 Technical Specification
# Version: 1.3
# Format: AI-first (structured for LLM parsing)

---
meta:
  name: index1
  version: "1.3"
  tagline: "AI-native project knowledge base"
  one_liner: "给 AI Agent 用的项目文档知识库，单文件存储，BM25 + 向量混合搜索，即时响应"
  
---
# 1. PROBLEM

## 1.1 Gap Analysis
| info_type | current_solution | status |
|-----------|------------------|--------|
| conversation_memory | claude-mem | ✅ solved |
| project_knowledge | ??? | ❌ gap |
| exact_code_location | built-in Grep/Glob | ✅ solved (no semantic) |

## 1.2 Competitor Issues
```yaml
qmd:
  issue_1: cold_start_latency_60s
  issue_2: fixed_length_chunking (ignores code structure)
  result: unusable_for_ai_agent

claude-mem:
  issue: only_indexes_conversations
  result: no_project_docs_support
```

---
# 2. ARCHITECTURE

## 2.1 System Overview
```
┌─────────────────────────────────────────────────────────────┐
│                      MCP Interface Layer                     │
├─────────────────────────────────────────────────────────────┤
│  docs_search │ docs_get │ docs_status │ docs_reindex │ ...  │
├─────────────────────────────────────────────────────────────┤
│                      Query Engine (Online)                   │
│  ┌─────────────┬─────────────┬─────────────┬─────────────┐  │
│  │ Ollama Embed│ BM25 (FTS5) │ Vec Search  │ RRF Fusion  │  │
│  │   ~100ms    │    ~5ms     │   ~10ms     │   instant   │  │
│  └─────────────┴─────────────┴─────────────┴─────────────┘  │
├─────────────────────────────────────────────────────────────┤
│                      Index Engine (Offline)                  │
│  watchdog → hash_diff → smart_chunk → batch_embed → SQLite  │
├─────────────────────────────────────────────────────────────┤
│                 SQLite Single File Storage                   │
│        (documents + chunks + FTS5 + sqlite-vec)              │
└─────────────────────────────────────────────────────────────┘
```

## 2.2 Database Schema

### documents table
| field | type | description |
|-------|------|-------------|
| id | INTEGER PK | auto increment |
| path | TEXT UNIQUE | relative file path |
| hash | TEXT | SHA256 for change detection |
| collection | TEXT | e.g., 'project-a' |
| doc_type | TEXT | markdown/python/rust/javascript/text/api |
| indexed_at | DATETIME | last index time |
| metadata | JSON | custom metadata |

### chunks table
| field | type | description |
|-------|------|-------------|
| id | INTEGER PK | auto increment |
| doc_id | INTEGER FK | references documents |
| content | TEXT | chunk text |
| section | TEXT | e.g., '## API Reference' |
| parent_section | TEXT | e.g., '# Risk Module' |
| chunk_index | INTEGER | order within document |
| tags | TEXT | comma-separated, indexed by FTS5 |
| embedding | BLOB | sqlite-vec vector |
| token_count | INTEGER | for response size control |
| summary | TEXT | auto-generated for progressive retrieval |

### chunks_fts (FTS5 virtual table)
```sql
CREATE VIRTUAL TABLE chunks_fts USING fts5(
    content, section, parent_section, tags,
    content=chunks,
    content_rowid=id
);

CREATE TRIGGER chunks_ai AFTER INSERT ON chunks BEGIN
    INSERT INTO chunks_fts(rowid, content, section, parent_section, tags)
    VALUES (new.id, new.content, new.section, new.parent_section, new.tags);
END;
```

## 2.3 Query Pipeline
```yaml
step_1:
  action: ollama_embed(query)
  latency: ~100ms
  output: query_vector[768]

step_2_parallel:
  bm25:
    action: FTS5_search(query_tokens)
    latency: ~5ms
    output: bm25_results[top_k * 2]
  vector:
    action: sqlite_vec_knn(query_vector)
    latency: ~10ms
    output: vector_results[top_k * 2]

step_3:
  action: RRF_fusion(bm25_results, vector_results)
  latency: <1ms
  formula: score = Σ 1/(k + rank_i)
  k: 60

step_4_optional:
  action: ollama_rerank(query, candidates)
  latency: ~200ms
  improvement: +7% precision

step_5:
  action: return_progressive(summaries)
  output: [{id, score, source, section, summary, relevance}]
```

---
# 3. SMART CHUNKING

## 3.1 Strategy by File Type
| file_type | strategy | example |
|-----------|----------|---------|
| markdown | split_by_heading | `# Risk > ## Liquidation` → 1 chunk |
| python | split_by_function_class | `def liquidate():` → 1 chunk |
| rust | split_by_fn_impl | `fn process()` → 1 chunk |
| javascript | split_by_function_class | `function calc()` → 1 chunk |
| api_doc | split_by_endpoint | `POST /api/liquidate` → 1 chunk |
| plain_text | split_by_paragraph | paragraph + 50 token overlap |

## 3.2 Chunk Metadata
```json
{
  "source": "docs/API.md",
  "section": "## Liquidation API",
  "parent_section": "# Risk Module",
  "tags": ["liquidation", "risk", "api", "清算"],
  "type": "api_doc",
  "token_count": 320
}
```

## 3.3 Code Parsing Strategy Decision
```yaml
problem: how_to_split_code_files_by_function_class
options:
  regex:
    pros: zero_deps, simple, fast
    cons: imprecise, fails_on_nested_structures
    accuracy: ~80%
  
  tree_sitter:
    pros: precise_AST, supports_50+_languages
    cons: requires_compiled_.so, adds_dependency
    accuracy: ~99%
  
  python_ast:
    pros: stdlib, zero_deps, precise_for_python
    cons: python_only
    accuracy: ~99% (python)

decision_by_phase:
  phase_1_mvp:
    markdown: regex (split by ## headings)
    python: stdlib ast module (precise, zero deps)
    rust: regex (fn/impl patterns, ~80% accuracy)
    javascript: regex (function/class patterns, ~80% accuracy)
    rationale: ship_fast, good_enough_for_mvp
  
  phase_2:
    rust: tree-sitter-rust
    javascript: tree-sitter-javascript
    typescript: tree-sitter-typescript
    rationale: improve_accuracy_after_validation

implementation_detail:
  python_chunker:
    tool: ast.parse() + ast.walk()
    extract: [FunctionDef, AsyncFunctionDef, ClassDef]
    preserve: docstrings, decorators
    code_lines: ~50
  
  regex_chunker:
    python_pattern: r'^(async\s+)?def\s+\w+|^class\s+\w+'
    rust_pattern: r'^(pub\s+)?(async\s+)?fn\s+\w+|^impl\s+'
    js_pattern: r'^(export\s+)?(async\s+)?function\s+\w+|^class\s+\w+'
    fallback: split_by_blank_lines_if_no_match

tradeoff_summary:
  │ language   │ phase_1      │ phase_2       │ accuracy │
  │────────────│──────────────│───────────────│──────────│
  │ markdown   │ regex        │ regex         │ 95%      │
  │ python     │ stdlib ast   │ stdlib ast    │ 99%      │
  │ rust       │ regex        │ tree-sitter   │ 80%→99%  │
  │ javascript │ regex        │ tree-sitter   │ 80%→99%  │
  │ typescript │ regex        │ tree-sitter   │ 80%→99%  │
```

---
# 4. MCP TOOLS

| tool | params | returns | purpose |
|------|--------|---------|---------|
| docs_search | query, collection?, top_k? | summary_list + confidence_scores | main search (hybrid) |
| docs_get | chunk_id | full_chunk + context | get details |
| docs_status | - | index_stats | health check |
| docs_reindex | collection?, path? | rebuild_status | manual reindex |
| docs_config | key?, value? | current_config | view/update config |

## 4.1 docs_search Response Format
```json
{
  "results": [
    {
      "id": "chunk_42",
      "score": 0.92,
      "source": "docs/API.md",
      "section": "## Liquidation API",
      "summary": "Progressive liquidation triggers when health factor < 1.2...",
      "relevance": "high"
    }
  ],
  "total": 15,
  "query_ms": 130
}
```

---
# 5. PERFORMANCE

## 5.1 Latency Comparison
| operation | qmd | claude-mem | grep | index1 |
|-----------|-----|------------|------|--------|
| keyword_search | ~5ms | ~10ms | <1ms | ~5ms |
| semantic_cold | ~60s | ~1s | N/A | ~3s |
| semantic_hot | ~60s | ~100ms | N/A | ~110ms |
| hybrid_search | ~120s | N/A | N/A | ~130ms |

## 5.2 Performance Claims Verification
```yaml
hybrid_search_130ms_breakdown:
  ollama_embed: 100ms  # query → vector, model already hot
  bm25_search: 5ms     # FTS5 full-text search
  vector_search: 10ms  # sqlite-vec brute-force KNN
  rrf_fusion: <1ms     # pure math, no I/O
  total: ~116ms        # rounded to ~130ms with overhead
  verification: ✅ numbers_add_up

"100x_faster_than_qmd"_claim:
  basis: qmd_hybrid_search_120s vs index1_130ms
  condition: qmd_cold_start (loads local LLM every query)
  qmd_behavior: 
    - uses node-llama-cpp for embeddings
    - loads model from disk on each search (~60s)
    - no daemon mode, no model caching
  index1_behavior:
    - ollama daemon keeps model in memory
    - first query ~3s (model load), subsequent ~100ms
  
  fair_comparison:
    cold_start: qmd 60s vs index1 3s → 20x faster
    hot_query: qmd 60s vs index1 130ms → 460x faster
    reason: qmd reloads model every time, index1 keeps hot
  
  marketing_claim: "100x faster" → ✅ valid (conservative)
  caveat: assumes qmd default behavior without optimization
```

## 5.3 Performance Gains
| optimization | before | after | technique |
|--------------|--------|-------|-----------|
| model_loading | 60s | 100ms | ollama_daemon_keep_hot |
| doc_embedding | per_query | zero | pre_compute_at_index |
| vector_search | network | 10ms | sqlite_vec_embedded |
| fusion | LLM | <1ms | RRF_math_formula |
| token_usage | 100% | 20-50% | progressive_retrieval |

---
# 6. OPTIMIZATION ANALYSIS

## 6.1 Tag Pre-filtering
```yaml
idea: filter_chunks_by_tag_before_vector_search
verdict: NOT_RECOMMENDED
reasons:
  - bottleneck_is_ollama_embed_100ms_not_vector_10ms
  - pre_filter_may_miss_semantically_related_chunks
  - implementation_complexity_high
when_useful: chunk_count > 100k
```

## 6.2 Hierarchical Index
```yaml
idea: search_section_summaries_first_then_drill_down
verdict: NOT_RECOMMENDED
reasons:
  - already_have_progressive_retrieval_on_response_side
  - doubles_index_maintenance_cost
  - may_miss_chunks_not_covered_by_section_summary
when_useful: single_search > 100ms
```

## 6.3 Query Cache
```yaml
idea: LRU_cache_for_frequent_queries
verdict: RECOMMENDED_FOR_PHASE_2
reasons:
  - simple_implementation_50_lines
  - bypasses_100ms_embed_bottleneck
  - developer_queries_highly_repetitive
  - cache_invalidation_easy_via_watchdog

implementation:
  L1_cache:
    key: (query, collection, top_k)
    hit_rate: 40-60%
    ttl: 10_minutes
    invalidation: file_change
  
  L2_cache:
    key: query_text
    value: embedding_vector
    hit_rate: 60-80%
    benefit: skip_100ms_embed_still_need_15ms_search

memory:
  per_entry: 4KB (3KB vector + 1KB summary)
  1000_entries: 4MB
```

---
# 7. TECH STACK

| component | choice | reason |
|-----------|--------|--------|
| language | Python | ecosystem, ollama_api_support |
| storage | SQLite + FTS5 + sqlite-vec | single_file, zero_deps |
| embedding | Ollama REST API | local, hot_daemon, model_swap |
| mcp | mcp-python SDK | official |
| file_watch | watchdog | mature |
| distribution | pip install index1 | standard |

## 7.1 Risk: sqlite-vec
```yaml
status: v0.1.6 (2024-11)
risk: api_breaking_changes
mitigation:
  - pin_version: "sqlite-vec==0.1.6"
  - wrap_with_DAL_layer
  - fallback_to_FAISS_if_abandoned
assessment: sufficient_for_target_scale (<10k chunks)
```

## 7.2 Risk: Ollama Hard Dependency
```yaml
problem: if_ollama_unavailable_index1_completely_unusable
impact: critical_single_point_of_failure

solution_1_graceful_degradation:
  trigger: ollama_connection_failed OR timeout > 3s
  behavior:
    - fallback_to_bm25_only_search
    - return_results_with_flag: { "mode": "bm25_only", "reason": "ollama_unavailable" }
    - log_warning: "向量搜索不可用，已降级为关键词搜索"
  benefit: tool_still_usable, user_not_blocked
  tradeoff: no_semantic_search (miss "风控" → "去杠杆" associations)

solution_2_startup_detection:
  on_init:
    - check: curl http://localhost:11434/api/version
    - if_fail: print_clear_error_message
    - suggest: "请运行 'ollama serve' 或 'brew services start ollama'"
  on_first_search:
    - lazy_check: only_when_vector_search_needed
    - cache_status: avoid_repeated_checks

solution_3_remote_api_fallback:
  config_option:
    embedding_provider: ollama | openai | anthropic | custom
  implementation:
    ollama:
      endpoint: http://localhost:11434/api/embed
      model: nomic-embed-text
      cost: free
    openai:
      endpoint: https://api.openai.com/v1/embeddings
      model: text-embedding-3-small
      cost: $0.02/1M tokens
    anthropic:
      endpoint: https://api.anthropic.com/v1/embed  # if available
      model: TBD
      cost: TBD
    custom:
      endpoint: user_defined
      headers: user_defined
  priority: ollama → remote_api → bm25_only

implementation_priority:
  P0_mvp: startup_detection + clear_error_message
  P1: graceful_degradation_to_bm25
  P2: remote_api_fallback_option
```

### Degradation Mode Behavior
```
┌─────────────────────────────────────────────────────────┐
│                    Query Flow                            │
├─────────────────────────────────────────────────────────┤
│  1. Check Ollama status (cached, refresh every 60s)     │
│                                                          │
│  IF ollama_available:                                    │
│     → embed(query) → BM25 + Vector → RRF → results      │
│     → return { mode: "hybrid", results: [...] }         │
│                                                          │
│  ELSE:                                                   │
│     → BM25 only → results                                │
│     → return { mode: "bm25_only",                       │
│                reason: "ollama_unavailable",             │
│                hint: "运行 ollama serve 启用语义搜索",    │
│                results: [...] }                          │
└─────────────────────────────────────────────────────────┘
```

### Config Example with Fallback
```yaml
# ~/.claude-index1/config.yaml

embedding:
  provider: ollama  # ollama | openai | custom
  
  ollama:
    endpoint: http://localhost:11434
    model: nomic-embed-text
    timeout: 3s
  
  openai:  # fallback option
    api_key: ${OPENAI_API_KEY}
    model: text-embedding-3-small
  
  fallback_to_bm25: true  # if all embedding providers fail
```

## 7.3 CJK Tokenization (Critical for Chinese/Japanese)
```yaml
problem: FTS5_default_unicode61_tokenizer_unusable_for_chinese
impact: 🔴 critical_if_target_users_include_chinese_projects
example:
  query: "清算接口"
  unicode61_behavior: treats_as_single_token, no_word_boundary
  result: only_exact_match_works, "清算" alone won't match

solutions:
  option_A_jieba_tokenizer:
    approach: custom_FTS5_tokenizer_with_jieba
    pros: precise_chinese_segmentation
    cons: complex_implementation, ~200_lines, jieba_dependency
    accuracy: 95%+
    
  option_B_trigram_tokenizer:
    approach: FTS5_trigram_tokenizer
    sql: "CREATE VIRTUAL TABLE chunks_fts USING fts5(content, tokenize='trigram')"
    pros: simple, works_for_any_language
    cons: fuzzy_matching, larger_index, slower
    accuracy: 80%
    
  option_C_vector_only_for_cjk:
    approach: detect_cjk → skip_bm25 → vector_search_only
    pros: simplest, zero_extra_deps
    cons: lose_keyword_precision_for_chinese
    accuracy: depends_on_embedding_model
    
  option_D_icu_tokenizer:
    approach: FTS5_with_ICU_extension
    pros: proper_unicode_segmentation
    cons: requires_icu_library, not_always_available
    accuracy: 90%

decision:
  phase_1_mvp: option_C (simplest, validate first)
  phase_2: option_B (trigram) OR option_A (jieba)
  
  implementation:
    detect_cjk: regex r'[\u4e00-\u9fff\u3040-\u309f\u30a0-\u30ff]'
    if_cjk_in_query:
      bm25_weight: 0.2  # reduce BM25 influence
      vector_weight: 0.8  # rely more on semantic
    else:
      bm25_weight: 0.5
      vector_weight: 0.5
```

## 7.4 Large File Handling
```yaml
problem: files_>1MB_may_cause_memory_issues_or_slow_indexing
threshold: 1MB (configurable)

strategy:
  small_file (<1MB):
    action: load_entire_file, chunk_normally
  
  large_file (1MB-10MB):
    action: stream_read, chunk_incrementally
    implementation: read_in_64KB_blocks, maintain_context_window
  
  huge_file (>10MB):
    action: skip_with_warning OR split_by_markers
    log: "Skipping large file: {path} ({size}MB). Set max_file_size to override."

config:
  max_file_size_mb: 10
  large_file_strategy: stream | skip | split
```

## 7.5 Document Deletion Sync
```yaml
problem: watchdog_detects_file_changes_but_deletion_handling_unclear

implementation:
  on_file_delete:
    1. watchdog_event: FileDeletedEvent
    2. lookup: SELECT id FROM documents WHERE path = ?
    3. cascade_delete:
       - DELETE FROM chunks WHERE doc_id = ?
       - DELETE FROM documents WHERE id = ?
    4. fts5_sync: automatic via trigger (chunks_fts content=chunks)
    5. vector_cleanup: sqlite-vec handles via foreign key

  on_directory_delete:
    action: delete_all_docs_with_path_prefix
    sql: DELETE FROM documents WHERE path LIKE 'deleted_dir/%'

  orphan_cleanup:
    trigger: on_startup OR weekly
    action: scan_for_docs_without_source_files
```

## 7.6 Error Recovery
```yaml
problem: index_interrupted_midway_leaves_inconsistent_state

scenarios:
  scenario_1_embed_failure:
    cause: ollama_crash_during_batch_embed
    state: some_chunks_have_null_embedding
    recovery:
      - on_startup: SELECT id FROM chunks WHERE embedding IS NULL
      - action: re-embed_incomplete_chunks
  
  scenario_2_write_failure:
    cause: disk_full_or_db_locked
    state: partial_transaction
    recovery:
      - sqlite_handles_via_WAL_journal
      - on_startup: automatic_rollback
  
  scenario_3_crash_during_reindex:
    cause: process_killed
    state: some_files_indexed, some_not
    recovery:
      - track: last_indexed_file_path in metadata table
      - on_startup: resume_from_last_checkpoint

implementation:
  checkpoint_table:
    CREATE TABLE index_progress (
      collection TEXT PRIMARY KEY,
      last_file TEXT,
      status TEXT,  -- 'in_progress' | 'completed' | 'failed'
      updated_at DATETIME
    );
  
  recovery_flow:
    1. check index_progress for incomplete jobs
    2. if found: prompt user "上次索引未完成，是否继续？"
    3. resume or restart based on user choice
```

## 7.7 Multi-Project Isolation
```yaml
current_design: collection_field_in_documents_table
status: schema_ready, CLI_interaction_needs_clarity

cli_commands:
  # List collections
  index1 collections
  
  # Index specific collection
  index1 index ./project-a --collection=project-a
  index1 index ./project-b --collection=project-b
  
  # Search within collection
  index1 search "清算" --collection=project-a
  
  # Search across all (default)
  index1 search "清算"
  
  # Delete collection
  index1 delete --collection=project-a

mcp_tools:
  docs_search:
    params:
      query: string (required)
      collection: string (optional, default=all)
      top_k: int (optional, default=5)

storage:
  option_A_single_db: all collections in one knowledge.db (current design)
  option_B_separate_db: ~/.claude-index1/{collection}.db per project
  
  decision: option_A for MVP (simpler), option_B if isolation needed later
```

## 7.8 Testing Strategy
```yaml
problem: spec_has_no_testing_plan
priority: 🟡 medium, but essential for reliability

phase_1_mvp_tests:
  unit_tests:
    - chunker: test_markdown_split, test_python_ast_split
    - search: test_bm25_query, test_vector_query, test_rrf_fusion
    - db: test_insert_chunk, test_delete_cascade
  
  integration_tests:
    - index_then_search: index_sample_docs → search → verify_results
    - ollama_mock: test_degradation_to_bm25
  
  coverage_target: 70% for core modules

phase_2_tests:
  - benchmark: measure_latency_on_1k/5k/10k_chunks
  - stress: concurrent_search_requests
  - cjk: chinese_tokenization_accuracy

test_data:
  location: tests/fixtures/
  contents:
    - sample_markdown.md (english + chinese)
    - sample_python.py (functions + classes)
    - sample_rust.rs
    - expected_chunks.json

ci_setup:
  tool: pytest + github_actions
  on_pr: run_unit_tests
  on_merge: run_full_suite + coverage_report
```

---
# 8. DEVELOPMENT ROADMAP

## Phase 1: MVP (2-3 weeks)
- [ ] SQLite + FTS5 + sqlite-vec storage
- [ ] Ollama embed (embeddinggemma)
- [ ] Smart chunking (Markdown + code)
- [ ] MCP Server (search + get + status)
- [ ] CLI (index + search + status)
- [ ] RRF fusion

## Phase 2: Polish (1-2 weeks)
- [ ] watchdog file monitoring
- [ ] Progressive retrieval
- [ ] Optional rerank
- [ ] Query cache (L1 + L2)
- [ ] Claude Code plugin packaging

## Phase 3: Launch (1 week)
- [ ] Model config
- [ ] One-click setup script
- [ ] README + demo GIF
- [ ] Performance benchmark report

## Code Estimate
| module | lines | notes |
|--------|-------|-------|
| search_engine | ~500 | FTS5 + sqlite-vec + RRF |
| chunker | ~300 | multi-filetype strategies |
| mcp_server | ~200 | 5 tools |
| cli | ~150 | index/search/status/config |
| cache | ~50 | L1 + L2 |
| other | ~350 | watchdog, ollama, config |
| **total** | **~1550** | Python only, no tests |

---
# 9. ADDITIONAL OPTIMIZATIONS (NEW)

## 9.1 Embedding Model Selection

### Model Comparison
| model | size | dim | latency | accuracy | context | best_for |
|-------|------|-----|---------|----------|---------|----------|
| EmbeddingGemma | 300MB | 768→128 | ~20ms | good | 2048 | low_resource/edge |
| nomic-embed-text | 274MB | 768→256 | ~50ms | high | 8192 | general_purpose |
| nomic-embed-text-v2-moe | 475MB | 768→256 | ~80ms | highest | 8192 | max_precision |

### Recommended Configurations
```yaml
lightweight_mode:
  model: EmbeddingGemma
  size: 300MB (quantized <200MB)
  dim: 768
  latency: ~20ms
  target: 8GB_laptop, mobile, edge_devices
  note: Google Gemma 3 based, optimized for on-device

standard_mode:
  model: nomic-embed-text
  size: 274MB
  dim: 768
  latency: ~50ms
  target: 16GB+_general_use
  note: best_latency_quality_ratio, long_context_8192

high_precision_mode:
  model: nomic-embed-text-v2-moe
  size: 475MB (305M active params)
  dim: 768
  latency: ~80ms
  target: 32GB+_workstation
  note: MoE_architecture, 100+_languages, highest_accuracy
```

### Config File Example
```yaml
# ~/.claude-index1/config.yaml

# Lightweight (8GB laptop)
embedding_model: embeddinggemma
embedding_dim: 768

# Standard (16GB+, recommended)
# embedding_model: nomic-embed-text
# embedding_dim: 768

# High precision (32GB+)
# embedding_model: nomic-embed-text-v2-moe
# embedding_dim: 768
```

### Selection Logic
```
if RAM < 12GB:
    use EmbeddingGemma (fast, lightweight)
elif need_max_precision:
    use nomic-embed-text-v2-moe (MoE, highest accuracy)
else:
    use nomic-embed-text (default, balanced)
```

## 9.2 Batch Query Optimization
```yaml
scenario: AI_agent_sends_multiple_related_queries
current: each_query_runs_full_pipeline
optimization:
  technique: batch_embed_multiple_queries
  benefit: amortize_ollama_call_overhead
  implementation:
    - collect_queries_within_100ms_window
    - single_ollama_batch_embed_call
    - parallel_search_for_each
  estimated_gain: 30-50%_for_burst_queries
```

## 9.3 Incremental Index Update
```yaml
current: reindex_entire_file_on_change
issue: slow_for_large_files
optimization:
  technique: diff_based_chunk_update
  implementation:
    - compute_chunk_hashes
    - only_reembed_changed_chunks
    - preserve_unchanged_embeddings
  complexity: medium
  benefit: 80%_faster_reindex_for_small_edits
  priority: P2 (after_MVP)
```

## 9.4 Connection Pooling for Ollama
```yaml
current: new_http_connection_per_request
optimization:
  technique: http_connection_pool
  implementation: httpx.AsyncClient with keep-alive
  benefit: save_10-20ms_per_request
  priority: P1
```

## 9.5 Async Query Execution
```yaml
current: sequential BM25 → vector → fusion
optimization:
  technique: asyncio.gather for parallel execution
  implementation:
    - async_fts5_query
    - async_vec_query  
    - await_both_then_fuse
  benefit: reduce_15ms_to_10ms (5ms savings)
  priority: P1
```

## 9.6 Pre-warm Common Queries
```yaml
scenario: first_query_after_cold_start_is_slow
optimization:
  technique: background_prewarm_on_startup
  implementation:
    - store_recent_N_queries
    - on_startup: embed_and_cache_top_queries
  benefit: instant_response_for_common_patterns
  priority: P2
```

---
# 10. KEY TECHNICAL DECISIONS SUMMARY

```yaml
┌────────────────────┬─────────────────────────┬─────────────────────────────────────┐
│     Decision       │      Recommendation     │              Rationale              │
├────────────────────┼─────────────────────────┼─────────────────────────────────────┤
│ Default embedding  │ nomic-embed-text        │ Best balance: 274MB, 50ms, 8192 ctx │
│ model              │                         │ EmbeddingGemma for low-resource     │
├────────────────────┼─────────────────────────┼─────────────────────────────────────┤
│ Code parsing       │ Phase 1: ast + regex    │ Control initial dependencies        │
│                    │ Phase 2: tree-sitter    │ Python ast=99%, regex=80% accuracy  │
├────────────────────┼─────────────────────────┼─────────────────────────────────────┤
│ CJK support        │ Phase 1: trigram +      │ Simplest viable solution            │
│                    │ vector-weighted search  │ Chinese relies on semantic search   │
├────────────────────┼─────────────────────────┼─────────────────────────────────────┤
│ Ollama fallback    │ Phase 1 必做            │ Critical user experience path       │
│                    │ Degrade to BM25-only    │ Tool must work without Ollama       │
├────────────────────┼─────────────────────────┼─────────────────────────────────────┤
│ Async framework    │ asyncio + httpx         │ MCP server is async by design       │
│                    │                         │ httpx for connection pooling        │
├────────────────────┼─────────────────────────┼─────────────────────────────────────┤
│ Distribution       │ pip install index1      │ Standard Python packaging           │
│                    │                         │ PyPI + GitHub releases              │
├────────────────────┼─────────────────────────┼─────────────────────────────────────┤
│ Storage            │ Single SQLite file      │ Zero external services              │
│                    │ FTS5 + sqlite-vec       │ Backup = copy file                  │
├────────────────────┼─────────────────────────┼─────────────────────────────────────┤
│ Search fusion      │ RRF (k=60)              │ <1ms, no LLM needed                 │
│                    │                         │ Proven effective in literature      │
├────────────────────┼─────────────────────────┼─────────────────────────────────────┤
│ Progressive        │ Summary → Detail        │ Save 50-80% tokens                  │
│ retrieval          │ with confidence scores  │ AI decides what to fetch            │
└────────────────────┴─────────────────────────┴─────────────────────────────────────┘
```

## Phase 1 MVP → 发布 v0.1.0
```yaml
完成即发布，不等后续功能:
  - [ ] SQLite + FTS5 + sqlite-vec 存储
  - [ ] nomic-embed-text 嵌入
  - [ ] Markdown + Python 分块
  - [ ] BM25 + 向量 + RRF 融合搜索
  - [ ] MCP server (search + get + status)
  - [ ] CLI (index + search + status)
  - [ ] Ollama 检测 + 报错提示
  - [ ] README + 安装说明

发布动作:
  1. pip install index1 可用
  2. GitHub release v0.1.0
  3. 发 Twitter/X 预告
  4. 收集反馈
```

## Phase 2 → 发布 v0.2.0
```yaml
根据用户反馈决定优先级:
  - [ ] watchdog 文件监听
  - [ ] 渐进式检索
  - [ ] 查询缓存
  - [ ] BM25 降级模式
  - [ ] 可选 rerank
```

## Phase 3 → 发布 v0.3.0 / v1.0.0
```yaml
打磨和传播:
  - [ ] CJK trigram 分词
  - [ ] tree-sitter 代码解析
  - [ ] demo GIF
  - [ ] 提交 awesome-agent-skills
```

---
# 11. SUMMARY

```yaml
core_value:
  - fills_claude_code_knowledge_gap
  - single_file_storage (backup/sync/git friendly)
  - 100x_faster_than_qmd
  - complements_claude_mem

key_differentiators:
  - AI-native: designed_for_agent_not_human_adaptation
  - smart_chunking: structure_aware_not_fixed_length
  - hybrid_search: BM25 + vector + RRF
  - progressive_retrieval: save_tokens_with_confidence_scores
  - configurable: 8GB_laptop_to_64GB_workstation

next_action: start_phase_1_mvp
```

---
# END OF SPECIFICATION
