# Lessons Learned

Append-only log of issues caught during builds. Check when planning future phases.

---

## Phase 8: Test Writing for LLM Pipeline (2026-03-10)

### L1. MagicMock does not work with framework operator overloading

**What happened:** The test-writing agent created `mock_chat_model()` using `MagicMock` for LangChain's `BaseChatModel`. When production code does `chain = PROMPT | model.with_structured_output(Schema)`, LangChain's `__ror__` wraps the MagicMock in a `RunnableLambda`, bypassing the mock's configured `invoke()` return value. All 19 mission generator tests failed with "Expected CategoryMissionsResponse, got MagicMock."

**Why it wasn't caught:** The test-writing agent either didn't run the tests or ran them without proper PYTHONPATH. The iterative-build skill didn't enforce running tests after writing them.

**The agent also wrote the fix but didn't apply it.** `conftest.py` contains `mock_mission_chain()` which patches `MISSION_PROMPT.__or__` correctly — but no tests use it.

**Lesson:** When mocking objects that participate in operator chains (`|`, `>>`, `@`, etc.), patch the operator source (the prompt/template), not the operand (the model). Always smoke-test mock helpers with one real test before writing a full test suite.

**Prevention:** Added to action plan — B1 (mandatory test execution), B3 (mock protocol verification in testing.md), B4 (antipattern entry).

---

### L2. chmod has no place in a non-security application

**What happened:** The agent added `chmod 600` calls to protect config files, database files, and marker files. This was a "proactive hardening" pattern copied from security-sensitive tools (SSH, GPG). But this app is a codebase onboarding tool — it doesn't manage secrets, and the config-file API key storage that motivated chmod was unnecessary since env vars are already the primary mechanism.

**Why it wasn't caught during implementation:** The Phase 1 and Phase 2 reviews accepted the chmod pattern as reasonable security practice. It wasn't questioned until the container blocked chmod and forced re-evaluation.

**Decision:** Remove all chmod calls. Remove config-file API key support. Env vars only for API keys. The security scanning module (`secrets.py`) already does the right thing — it reports secret locations without capturing secret values.

**Lesson:** Question whether security patterns match the application's actual threat model. "chmod everything" is appropriate for SSH; it's over-engineering for an analysis tool that should never touch user secrets.

---

### L3. Test phases must verify tests actually pass

**What happened:** Phase 8 was marked complete with 87 failing tests (later reduced to 28 after partial fixes). The phase plan included "Verify: pytest tests/test_file.py -v — all pass" per task, but this was aspirational, not enforced.

**Root cause:** The iterative-build skill's test phase rules don't mandate running tests. The agent treated "Verify" as documentation, not as a gate.

**Lesson:** Test phases need stricter gates than implementation phases. A test that doesn't run is worse than no test — it creates false confidence. The phase cannot be marked complete until all tests pass AND the full suite shows no regressions.

**Prevention:** Added to action plan — B1 (mandatory test execution verification).

---

### L4. Container environment must be verified before test writing

**What happened:** The test-writing agent started writing tests without verifying that the test environment worked. PYTHONPATH wasn't set, causing all imports to fail. The agent may have worked around this silently or not noticed.

**Lesson:** First action in any test phase: run `pytest --collect-only -q` on existing tests. If collection fails, stop and fix the environment before writing anything.

**Prevention:** Added to action plan — B1 rule 3 (environment setup before first test), A1 (PYTHONPATH auto-detection in container).

---

### L5. Deferred items need follow-through

**What happened:** The chmod concern was flagged in `deferred.md` during Phase 8 review as "revisit before Phase 9." But it took a debugging session on failing tests to actually force the discussion. The deferred item sat as a note rather than triggering action.

**Lesson:** Deferred items marked "before Phase N" should be resolved during Phase N planning, not deferred further. The `/plan:phase` command should check `deferred.md` and block planning until deferred items for that phase are resolved.

---

## Post-Phase 12: Outside-Process Changes (2026-03-17)

### L6. Mechanical debt fixes don't need full phase overhead

**What happened:** After Phase 12 review, 6 tech debt items were fixed in 4 commits outside the `/plan:phase` → `/plan:build` process. All were mechanical: moving imports to module level, replacing `except Exception` with specific types, swapping a mutable default for `functools.partial`, wiring an existing tested function into the pipeline. 890 tests continued passing.

**Why it worked:** The fixes were small, independent, had clear specs in `technical_debt.md`, and were fully covered by existing tests. No new behavior was introduced — only existing behavior was corrected.

**Why documentation still matters:** The state.md, technical_debt.md, and deferred.md got out of sync because the changes weren't tracked. A documentation sync session was needed to reconcile what was in the code vs. what the planning docs described. This creates confusion about project state.

**Lesson:** For mechanical debt fixes: skip the full phase, but immediately update `state.md` and `technical_debt.md` after committing. One session log entry per commit batch is sufficient. The planning overhead is in the *documentation*, not the *planning process itself*.
