# Change Request: CR-002

> Created: 2026-03-17
> Status: Approved
> Classification: Modification
> Related Feature: FEATURE-049-F

## Request Details

**Requestor:** Project Owner
**Date Received:** 2026-03-17
**Priority:** High

## Description

Replace frontmatter-embedded metadata with a centralized `.kb-index.json` registry. The current approach embeds YAML frontmatter directly inside markdown files, which fundamentally fails for non-textual knowledge (images, videos, PDFs, binary files). The new approach stores metadata in a hidden JSON registry file that is locally-scoped and works uniformly for ALL file types.

**Key Design Decisions (from research on OpenClaw, Mem0, LangChain, CrewAI):**
- Industry consensus: **separate metadata from content** — files hold content, an index holds metadata
- OpenClaw pattern: files as canonical source of truth, sidecar index for metadata (rebuildable)
- Mem0 pattern: structured JSON metadata per record, extensible custom fields

**User-Specified Variations:**
1. **Hidden file:** `.kb-index.json` (dot-prefixed, not visible in normal file listings)
2. **Locally-scoped:** Each `.kb-index.json` indexes only the files and subfolders in its directory
3. **Folder metadata:** Folders can also have metadata entries (title, description, tags)
4. **Description attribute:** New `description` field (< 100 words) per entry

## Business Justification

The current frontmatter approach has a fundamental limitation: it cannot attach metadata to non-textual files (images, videos, PDFs). This means binary knowledge assets get NO tags, NO description, and are invisible to AI agent classification/retrieval. The Knowledge Base is designed to hold all types of knowledge, not just markdown. A unified metadata approach is required for the AI Librarian to properly organize and classify all file types.

## Impact Analysis

### Classification

**Type:** Modification to FEATURE-049-F

**Reasoning:**
- Same feature scope (KB metadata management), same users, same workflow
- Replaces an internal implementation detail (where metadata lives) — API shape can remain compatible
- No new UI pages or endpoints needed — existing file list/detail views continue to work
- The `frontmatter` field in API responses becomes populated from `.kb-index.json` instead of YAML parsing
- Backwards compatible: existing frontmatter in markdown files can be migrated to the registry

### Comparison Matrix

| Aspect | Current (Frontmatter) | Proposed (.kb-index.json) |
|--------|-----------------------|---------------------------|
| File types supported | Markdown only | ALL file types |
| Metadata location | Embedded in file content | Separate hidden JSON file |
| Folder metadata | ❌ Not supported | ✅ Supported |
| Description field | ❌ Not available | ✅ < 100 words |
| AI agent CRUD | Parse YAML → modify → rewrite file | Read JSON → update key → write JSON |
| Non-text files | No metadata at all | Full metadata support |
| Scope | Global (single in-memory index) | Local per-directory |
| Git-friendliness | Metadata diffs mixed with content diffs | Clean separation — metadata diffs in `.kb-index.json` |
| Human readability | YAML in file (readable) | JSON file (readable) |

### Conflict Analysis

**Conflicts Found:** Yes (internal — all within FEATURE-049-F)

| # | Type | Affected Component | Severity | Classification | Description |
|---|------|--------------------|----------|----------------|-------------|
| 1 | Code | `_parse_frontmatter()` | Medium | Replace | Parsing logic changes from YAML-in-file to JSON-from-index |
| 2 | Code | `_serialize_frontmatter()` | Medium | Remove | No longer needed — metadata not written into files |
| 3 | Code | `_auto_populate_frontmatter()` | Low | Rename/Refactor | Becomes `_auto_populate_index_entry()` — same logic, different target |
| 4 | Code | `_frontmatter_index` dict | Low | Refactor | Populated from `.kb-index.json` files instead of YAML parsing |
| 5 | Code | `create_file()` / `update_file()` | Medium | Refactor | Metadata written to index instead of serialized into file content |
| 6 | Spec | AC-049-F-12a | Medium | Update | "YAML frontmatter" → "index entry in `.kb-index.json`" |
| 7 | Spec | AC-049-F-12b | Medium | Update | Non-markdown files now GET metadata (in index) |
| 8 | Spec | AC-049-F-12c | Low | Update | "Existing frontmatter preserved" → "Existing index entries preserved" |
| 9 | Skill | `x-ipe-tool-kb-librarian` | Medium | Update | Frontmatter generation → index entry generation |
| 10 | Tests | `test_kb_service.py` | Medium | Update | ~38 frontmatter references need updating |

**Resolution Strategy:** All conflicts are internal to FEATURE-049-F and resolve through the same CR workflow. No cross-feature conflicts.

### Affected Components

| Component | Impact Level | Details |
|-----------|--------------|---------|
| `src/x_ipe/services/kb_service.py` | High | New `_read_kb_index()`, `_write_kb_index()`, `_get_index_entry()`, `_set_index_entry()` methods; refactor frontmatter methods; add description field |
| `src/x_ipe/routes/kb_routes.py` | Low | API response shape preserved — `frontmatter` field populated from index |
| `src/x_ipe/static/js/features/kb-browse-modal.js` | None | No changes — consumes same API JSON shape |
| `.github/skills/x-ipe-tool-kb-librarian/SKILL.md` | Medium | Update procedure: generate index entries instead of frontmatter |
| `tests/test_kb_service.py` | Medium | Update tests to use index-based metadata |
| `specification.md` | Medium | Update ACs for index-based metadata, add new ACs for folders/description |
| `technical-design.md` | Medium | New section for `.kb-index.json` architecture |

### Dependencies

- FEATURE-049-F infrastructure (intake service, APIs, UI) — ✅ already implemented
- CR-001 (x-ipe-tool-kb-librarian skill) — ✅ already completed

## Proposed Schema

```json
{
  "version": "1.0",
  "entries": {
    "meeting-notes.md": {
      "title": "Clienteling AI Tool Meeting Notes",
      "description": "Summary of the alignment meeting on AI-assisted clienteling system requirements and priorities.",
      "tags": {
        "domain": ["design"],
        "lifecycle": ["requirement"]
      },
      "author": "yzhang",
      "created": "2026-03-16",
      "type": "markdown"
    },
    "architecture.png": {
      "title": "System Architecture Diagram",
      "description": "High-level architecture showing the clienteling AI tool components and data flow.",
      "tags": {
        "domain": ["infrastructure"],
        "lifecycle": ["design"]
      },
      "author": "unknown",
      "created": "2026-03-16",
      "type": "image"
    },
    "subfolder-name/": {
      "title": "Design Documents",
      "description": "Collection of design artifacts for the clienteling AI tool.",
      "tags": {
        "domain": ["design"],
        "lifecycle": ["design"]
      }
    }
  }
}
```

**Schema Rules:**
- Folder entries end with `/` to distinguish from files
- `type` field for files: `markdown`, `image`, `video`, `pdf`, `document`, `other`
- `description`: max 100 words, plain text
- `tags`: same 2D taxonomy (lifecycle × domain) from `kb-config.json`
- Missing entries = file has no metadata (default behavior for new files)

## Proposed Approach

1. Update `specification.md`: Modify ACs for index-based metadata, add ACs for folder metadata + description
2. Update `technical-design.md`: Add `.kb-index.json` architecture section
3. Implement in `kb_service.py`: New index read/write methods, refactor frontmatter methods
4. Update `x-ipe-tool-kb-librarian` skill: Generate index entries instead of frontmatter
5. Migration: One-time migration of existing frontmatter → `.kb-index.json` entries
6. Update tests

## Mockup Impact

**UI/UX Change:** Minimal — the API response shape is preserved. The UI will gain:
- Folder metadata display (title, description in folder views)
- Description field display in file detail views
- Non-markdown files now show tags/metadata (previously empty)

## Approval

- [x] Classification approved (DAO-confirmed)
- [x] Conflict analysis reviewed (10 internal conflicts, all resolvable)
- [x] Approach approved (DAO-confirmed)
- [x] Ready for execution

## Execution

**Classification Result:** modification
**Conflicts Resolution:** All 10 conflicts are internal to FEATURE-049-F — resolved through spec update + code refactor in downstream tasks

**Actions Taken:**
- [x] CR document created in feature folder
- [x] Conflict analysis completed and documented
- [x] Specification Version History updated with CR link
- [ ] Next task type set: x-ipe-task-based-feature-refinement

## Links

- Feature Specification: [specification.md](./specification.md)
- Previous CR: [CR-001.md](./CR-001.md)
- Research: OpenClaw memory architecture, Mem0 metadata model, LangChain/CrewAI vector store patterns

---

## Change History

| Date | Change | By |
|------|--------|----|
| 2026-03-17 | CR created | Zephyr 🌬️ |
