# Jinja2 Migration — Implementation Plan

> **Status:** Phases 1–2 complete, merged to main (Feb 2026). Phase 3+ candidates listed at bottom.

_Created 9 Feb 2026._

## Overview

Migrate `bristlenose/stages/render_html.py` (~1883 lines) from `_w = parts.append` inline HTML to Jinja2 templates. The goal is identical HTML output at every step — no visual changes, no behavioural changes.

**Design doc:** `docs/private/frontend-evolution.md` (long-term roadmap, framework analysis, SaaS vision)
**This doc:** concrete implementation steps for the Jinja2 extraction

---

## Current state (Phase 0 complete)

Phase 0 is committed on the `jinja2-migration` branch. It provides:

| What | Where |
|------|-------|
| Worktree | `/Users/cassio/Code/bristlenose_branch jinja2-migration/` |
| Jinja2 dependency | `pyproject.toml` — `jinja2>=3.1` |
| Comparison script | `scripts/compare-render.sh` |
| Baseline HTML | `_comparison/baseline/` (7 files) |
| Parity tests | `tests/test_jinja2_parity.py` (12 tests) |

**render_html.py is untouched.** Phase 0 only added infrastructure.

### Comparison workflow (use for every step)

```bash
cd "/Users/cassio/Code/bristlenose_branch jinja2-migration"

# 1. Make the change
# 2. Run parity tests
.venv/bin/python -m pytest tests/test_jinja2_parity.py -v

# 3. Visual comparison (renders, diffs, opens browser if different)
scripts/compare-render.sh

# 4. If good: update baseline and commit
scripts/compare-render.sh --baseline

# 5. If bad: revert
git checkout -- bristlenose/stages/render_html.py
```

### Before starting Phase 1

If pending feature branches have merged into main since Phase 0:

```bash
cd "/Users/cassio/Code/bristlenose_branch jinja2-migration"
git rebase main
.venv/bin/pip install -e '.[dev]'
scripts/compare-render.sh --baseline   # re-capture baseline from updated code
.venv/bin/python -m pytest tests/ -x   # verify everything passes
```

---

## render_html.py structure (reference)

Three pages are generated by three entry points:

| Page | Function | Lines | JS set |
|------|----------|-------|--------|
| Report | `render_html()` | ~178–510 | 18 modules |
| Transcript | `_render_transcript_page()` | ~694–920 | 5 modules |
| Codebook | `_render_codebook_page()` | ~944–1017 | 4 modules |

All three use the same pattern:
```python
parts: list[str] = []
_w = parts.append
_w("<!DOCTYPE html>")
# ... hundreds of lines ...
path.write_text("\n".join(parts), encoding="utf-8")
```

Key helper functions (return HTML strings, no side effects). Line numbers updated after Phase 2:

| Function | Lines | What |
|----------|-------|------|
| `_footer_html()` | ~1119 | Page footer → `footer.html` template |
| `_format_quote_html()` | ~1179 | Single blockquote → `quote_card.html` template |
| `_quote_badges()` | ~1221 | Sentiment/intent badge spans |
| `_build_sentiment_html()` | ~1254 | Histogram → `sentiment_chart.html` template |
| `_build_rewatch_html()` | ~1417 | Friction points → `friction_points.html` template |
| `_build_task_outcome_html()` | ~1514 | User journeys → `user_journeys.html` template |
| `_build_coverage_html()` | ~1562 | Coverage → `coverage.html` template |
| `_write_player_html()` | ~1505 | Player → `player.html` template |

### Three layers of duplication

1. **Document shell** (DOCTYPE through `<article>`) — repeated 3 times. Near-identical except: `data-theme` attribute, `<title>`, CSS path (`assets/` vs `../assets/`)

2. **Report header** (logo + logotype + project name + doc title) — repeated 3 times. Near-identical except: `assets/` vs `../assets/` prefix, doc title ("Research report" / "Session transcript" / "Codebook"), meta-right content

3. **Footer** — already a function (`_footer_html()`), but hardcodes `assets/` paths. Transcript pages are in `sessions/` and need `../assets/`. This is actually a latent bug (logo paths broken on transcript pages)

---

## Phase 1 — Step-by-step extraction

### Step 1a: Jinja2 Environment setup

**What:** Add a module-level `jinja2.Environment` with `FileSystemLoader` pointing at `bristlenose/theme/templates/`.

**Where:** `render_html.py`, near the top (after the CSS/JS loading section, around line 155).

**Code:**
```python
import jinja2

_TEMPLATE_DIR = _THEME_DIR / "templates"
_jinja_env = jinja2.Environment(
    loader=jinja2.FileSystemLoader(str(_TEMPLATE_DIR)),
    autoescape=False,  # We manage escaping ourselves for now
    keep_trailing_newline=True,
)
```

**Why autoescape=False:** The current code uses `_esc()` (wrapping `html.escape()`) at the point of use. Switching to Jinja2 autoescaping would require auditing every `{{ }}` in every template to add `|safe` where raw HTML is intentional. That's a large, risky change. Better to keep manual escaping during migration and switch to autoescaping as a separate, later step.

**Risk:** Near zero. Adds an import and a module-level object. No existing behaviour changes.

**Template dir:** `bristlenose/theme/templates/` already exists (holds CSS files). Jinja2 `.html` templates will sit alongside them. This matches the atomic design system's directory structure.

---

### Step 1b: Footer extraction

**What:** Move `_footer_html()` into `footer.html` template. Parameterise the assets prefix.

**Current code** (lines 1160–1193):
```python
def _footer_html() -> str:
    from bristlenose import __version__
    return (
        '<footer class="report-footer">'
        '<div class="footer-left">'
        '<picture class="footer-logo-picture">'
        '<source srcset="assets/bristlenose-logo-dark.png" ...'
        ...
    )
```

**Template:** `bristlenose/theme/templates/footer.html`
```html
<footer class="report-footer">
<div class="footer-left">
<picture class="footer-logo-picture">
<source srcset="{{ assets_prefix }}bristlenose-logo-dark.png" media="(prefers-color-scheme: dark)">
<img class="footer-logo" src="{{ assets_prefix }}bristlenose-logo.png" alt="">
</picture>
<span class="footer-logotype">Bristlenose</span>&#8194;<a class="footer-version" href="https://github.com/cassiocassio/bristlenose">version {{ version }}</a>
</div>
<div class="feedback-links">
<a class="footer-link" href="https://github.com/cassiocassio/bristlenose/issues/new" target="_blank" rel="noopener">&#128027; Report a bug</a>
<span class="footer-link-sep">&middot;</span>
<a class="footer-link feedback-trigger" role="button" tabindex="0">&#9825; Feedback</a>
</div>
<a class="footer-keyboard-hint" role="button" tabindex="0"><kbd>?</kbd> for Help</a>
</footer>
```

**Updated Python:**
```python
def _footer_html(assets_prefix: str = "assets/") -> str:
    from bristlenose import __version__
    tmpl = _jinja_env.get_template("footer.html")
    return tmpl.render(version=__version__, assets_prefix=assets_prefix)
```

**Call sites updated:**
- Report (line 652): `_w(_footer_html())` — unchanged (default `assets/`)
- Codebook (line 1136): `_w(_footer_html())` — unchanged (default `assets/`)
- Transcript (line ~1008): `_w(_footer_html("../assets/"))` — fixes the latent bug

**Risk:** Very low. Self-contained function, return type unchanged (str). Fixes a real bug (transcript footer logos).

**Verification:**
- Parity test: report + codebook output should be identical
- Transcript pages: will have a diff (the bug fix — `../assets/` in footer logos)
- Visual: open both, verify footer looks correct on all 3 page types

---

### Step 1c: Document shell extraction

**What:** Extract the `<!DOCTYPE html>` through `<article>` boilerplate into `document_shell_open.html` and the `</article></body></html>` into inline (too small to template).

**Template:** `bristlenose/theme/templates/document_shell_open.html`
```html
<!DOCTYPE html>
<html lang="en"{% if color_scheme in ['light', 'dark'] %} data-theme="{{ color_scheme }}"{% endif %}>
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="color-scheme" content="light dark">
<title>{{ title }}</title>
<link rel="stylesheet" href="{{ css_href }}">
</head>
<body>
<article>
```

**Parameters:** `color_scheme`, `title`, `css_href`

**Three call sites:**
- Report (line 222): `title=project_name`, `css_href="assets/bristlenose-theme.css"`
- Transcript (line 886): `title="Session 1: m1, p1 — Project"`, `css_href="../assets/bristlenose-theme.css"`
- Codebook (line 1070): `title="Codebook — Project"`, `css_href="assets/bristlenose-theme.css"`

**Each replaces ~14 lines of `_w()` calls** with:
```python
_w(_jinja_env.get_template("document_shell_open.html").render(
    color_scheme=color_scheme,
    title=_esc(project_name),
    css_href="assets/bristlenose-theme.css",
))
```

**Risk:** Low. Pure boilerplate, one conditional (`data-theme`).

---

### Step 1d: Report header extraction

**What:** Extract the logo + logotype + project name + doc title block into `report_header.html`.

**Template:** `bristlenose/theme/templates/report_header.html`
```html
<div class="report-header">
<div class="header-left">
{% if has_logo %}
{% if has_dark_logo %}
<picture>
<source srcset="{{ assets_prefix }}bristlenose-logo-dark.png" media="(prefers-color-scheme: dark)">
<img class="report-logo" src="{{ assets_prefix }}bristlenose-logo.png" alt="Bristlenose logo">
</picture>
{% else %}
<img class="report-logo" src="{{ assets_prefix }}bristlenose-logo.png" alt="Bristlenose logo">
{% endif %}
{% endif %}
<span class="header-title"><span class="header-logotype">Bristlenose</span>&#8195;<span class="header-project">{{ project_name }}</span></span>
</div>
<div class="header-right">
<span class="header-doc-title">{{ doc_title }}</span>
{% if meta_right %}
{{ meta_right }}
{% endif %}
</div>
</div>
<hr>
```

**Parameters:** `assets_prefix`, `has_logo`, `has_dark_logo`, `project_name`, `doc_title`, `meta_right` (raw HTML string for the meta line — different per page)

**Three call sites:**
- Report: `doc_title="Research report"`, `meta_right='<span class="header-meta">...'`
- Transcript: `doc_title="Session transcript"`, `meta_right='<span class="header-meta">...'`
- Codebook: `doc_title="Codebook"`, `meta_right=None`

**Each replaces ~40 lines of `_w()` calls.**

**Risk:** Low. One conditional (logo dark/light), parameterised assets prefix.

---

### Step 1e: Quote card extraction

**What:** Move `_format_quote_html()` + `_quote_badges()` into `quote_card.html`.

This is the highest-value extraction — the quote card is rendered ~57 times in the ikea test project. It has:
- Conditional timecode linking (clickable if video exists)
- Conditional researcher context prefix
- Sentiment/intent badges with backward compatibility
- Hide/edit/star action buttons
- Data attributes for JS interactivity

**Template:** `bristlenose/theme/templates/quote_card.html` (~40 lines of HTML)

**Python changes:** `_format_quote_html()` becomes a thin wrapper that prepares a context dict and calls `tmpl.render()`. `_quote_badges()` either becomes a Jinja2 macro or stays as a Python helper called before the template.

**Risk:** Medium. Most complex component so far. But it's already a standalone function with clear inputs/outputs. The comparison script will catch any regressions.

---

## Phase 2 — Report body extraction (9 Feb 2026)

Extracted all remaining inline HTML from the report page's `render_html()` function. Data preparation stays in Python; HTML rendering moved to Jinja2 templates. Worked from most self-contained to most reused.

| Step | Template | What | Lines saved |
|------|----------|------|-------------|
| 2a | `player.html` | Static popout video player page | ~120 |
| 2b | `sentiment_chart.html` | Horizontal-bar histogram | ~30 |
| 2c | `friction_points.html` | Rewatch/friction points list | ~30 |
| 2d | `user_journeys.html` | Task outcome table | ~25 |
| 2e | `coverage.html` | Transcript coverage disclosure | ~45 |
| 2f | `toolbar.html` | Search, codebook, tag filter, view switcher, export | ~110 |
| 2g | `session_table.html` | Session summary table | ~20 |
| 2h | `toc.html` | Three-column table of contents | ~40 |
| 2i | `content_section.html` | Shared template for sections and themes (dedup) | ~50 |

**Result:** `render_html.py` reduced from 1987 → 1616 lines (−371). Combined with Phase 1 (−170), the file is down ~540 lines from the original ~1883.

**Pattern:** Each helper function (`_build_sentiment_html`, `_build_rewatch_html`, etc.) keeps its data logic but delegates HTML assembly to `tmpl.render()`. Static blocks (toolbar, player) are parameter-free templates. The `content_section.html` template is shared by both sections and themes — a true dedup.

---

## Template directory layout (after Phase 2)

```
bristlenose/theme/templates/
├── report.css               # existing
├── transcript.css           # existing
├── print.css                # existing
├── content_section.html     # Step 2i — shared sections/themes
├── coverage.html            # Step 2e
├── document_shell_open.html # Step 1c
├── footer.html              # Step 1b
├── friction_points.html     # Step 2c
├── player.html              # Step 2a
├── quote_card.html          # Step 1e
├── report_header.html       # Step 1d
├── sentiment_chart.html     # Step 2b
├── session_table.html       # Step 2g
├── toc.html                 # Step 2h
├── toolbar.html             # Step 2f
└── user_journeys.html       # Step 2d
```

## What remains (Phase 3+ candidates)

- **Transcript page body** (`_render_transcript_page`) — segments loop, back-link nav, JS bootstrap. Already uses Phase 1 templates for shell/header/footer
- **Codebook page body** (`_render_codebook_page`) — minimal HTML (mostly JS-populated). Already uses Phase 1 templates
- **Report JS bootstrap** (lines 497–527) — `<script>` block with JSON data injection. Could become a template, but mixing JS vars with Jinja2 `{{ }}` needs care
- **Autoescaping migration** — switch from `autoescape=False` + manual `_esc()` to Jinja2 `autoescape=True` + `|safe` annotations. Separate effort, lower priority

---

## Gotchas to watch for

1. **Autoescaping is OFF.** We use `_esc()` manually. Don't pass unescaped user text into `{{ }}` — always escape in Python before passing to the template, or use `{{ var | e }}` in the template.

2. **Whitespace matters.** Jinja2 adds newlines around `{% %}` blocks by default. The current `_w()` code puts each element on its own line joined by `\n`. Match this — use `{%- -%}` trim markers if needed to avoid extra blank lines.

3. **Unicode characters.** The current code uses `\u201c` (left quote), `\u201d` (right quote), `\u2003` (em space), `\u00a0` (nbsp), etc. In Jinja2 templates, use the literal Unicode characters or HTML entities. The comparison script will catch mismatches.

4. **Template caching.** `jinja2.Environment` caches compiled templates by default. This is fine for production but during development, if you edit a `.html` template and re-run without restarting Python, the old version may be served. The `auto_reload=True` default handles this (checks file mtime).

5. **Dict key naming conflicts.** Jinja2's attribute lookup resolves `group.items` as `dict.items` (the method) before `dict["items"]` (the key). Never use `items`, `keys`, `values`, `get`, `update`, `pop` as dict keys passed to templates. This caused a runtime crash in Phase 2 (`friction_points.html` used `group.items` — fixed by renaming to `group.entries`).

6. **Ruff will not lint `.html` files.** Template syntax errors will only be caught at runtime. The parity tests and comparison script are the safety net.

6. **hatch build artifacts.** `pyproject.toml` already includes `bristlenose/theme/**/*.css` and `bristlenose/theme/**/*.js` in artifacts. We need to add `bristlenose/theme/**/*.html` so the templates are included in the wheel. **This was not done in Phase 0 — do it in Step 1a.**

---

## Rollback strategy

At any point during Phase 1:

- **Single step rollback:** `git revert HEAD` (each step is one commit)
- **Full Phase 1 rollback:** `git reset --hard phase0` (tag the Phase 0 commit first)
- **Abandon the branch:** `git worktree remove "/Users/cassio/Code/bristlenose_branch jinja2-migration" && git branch -D jinja2-migration`

The comparison baseline is always available for verification.
