# Agent State

## CRITICAL DATA BUG — DATE RANGE IS WRONG — 5 MONTHS NOT 3 DAYS
- We have been saying "3.3 days of data (Feb 10-13)" — THIS IS WRONG
- `first_seen` on the sessions table is the PUSH/INGESTION date, NOT the actual activity date
- The REAL activity dates come from `messages.timestamp` which spans Sep 12, 2025 to Feb 13, 2026 (~5 MONTHS)
- Daily breakdown by message timestamp shows: Sep 2025 (17 sessions), Oct 2025 (63), Jan 2026 (92), Feb 1-13 (139 sessions), 56,812 total messages
- Analysis agent MUST: use `MIN(m.timestamp)::date` and `MAX(m.timestamp)::date` from messages (joined to sessions) for the true date range
- Update overview.json `date_range`, `analysis_period`, and ALL references to "3.3 days" or "Feb 10-13"
- The daily heatmap, hourly charts, and learning curves should all use message timestamps for the X-axis dates
- This changes the entire narrative — it's not a snapshot, it's a 5-month longitudinal study
- Deduct 3 points if the dashboard still says "3 days" or uses first_seen as the activity date

## CRITICAL BUG — DASHBOARD IS BROKEN — FIX IMMEDIATELY
- The dashboard is showing a BLACK SCREEN due to two crashes. This is the HIGHEST priority.
- Hero.tsx:133 — `summaryItems.map is not a function` because `executive_summary` in overview.json changed from an array to a string. Fix: guard with `Array.isArray()`, if string split by `. ` into array, if null default to `[]`.
- QualitativeInsights.tsx — `Cannot convert undefined or null to object` at `Object.entries(intent_distribution)` because `intent_distribution` is null/undefined. Fix: guard all `Object.entries()` calls with `|| {}` fallback.
- RULE FOR ALL FUTURE CODE: EVERY `.map()`, `Object.entries()`, `Object.keys()` call on data from JSON MUST have a null guard. Data shape can change between iterations. Use `Array.isArray(x) ? x : []` for arrays, `x || {}` for objects.
- The frontend agent MUST verify the dashboard loads without crashes after every change. Check that TypeScript types match the actual JSON shape.
- Deduct 3 points from reviewer score if the dashboard has a black screen / crash.

## HARD CONSTRAINT — DO NOT IGNORE
- ALL analysis MUST filter to org LIKE 'pratilipi%' ONLY
- Update compute_dashboard_data.py and ALL scripts to add WHERE s.org LIKE 'pratilipi%'
- Re-run analysis to regenerate all JSON files with this filter
- This is a privacy constraint — do NOT analyze other orgs' data

## DATA VALIDATION — CHECK DATE RANGES
- The user suspects we are only analyzing 3-4 days of data but there should be MORE
- Analysis agent: RUN this query first and print results: SELECT org, COUNT(*), MIN(first_seen)::date, MAX(last_updated)::date FROM sessions WHERE org LIKE 'pratilipi%' GROUP BY org
- Verify the full date range is being used — do NOT accidentally filter by date
- If the local DB only has a subset, flag it in the output and AGENT_STATE.md
- Make sure overview.json reflects the actual date range and session count

## DATA CAVEAT — COMMIT DETECTION IS UNRELIABLE — DROP IT
- Commit detection is a noisy signal — STOP using it entirely
- Not all developers run git commit/push through the AI CLI — many commit manually
- Do NOT show commit counts, commit rates, "shipped vs wasted", or any commit-based metric
- Remove all commit-related insights, charts, and framing from the dashboard
- Do NOT use commits to judge developer productivity or waste

## SESSION TIME = ASSISTANT RESPONSE TIME (NOT WALL CLOCK)
- Session duration MUST be calculated as the SUM of assistant response times (time between user message and assistant reply)
- Do NOT use first_seen to last_updated — that's wall clock time and includes idle/abandoned periods
- A 67-hour "session" is a dev who left and came back days later — NOT 67 hours of work
- For each message pair, compute: assistant_timestamp - user_timestamp = response time
- Sum all response times in a session = actual AI interaction time
- If timestamps are not granular enough, use inter-message gaps < 30 minutes as a fallback
- Discard gaps > 30 minutes as idle/abandoned
- ALL session duration metrics, charts, and insights must use this methodology
- Add a tooltip/caveat explaining the methodology: "Session time = sum of AI response times, excluding idle gaps > 30 min"
- Deduct 2 points from reviewer satisfaction score for using wall clock time

## STRICTLY NO EMOJIS
- Do NOT use any emojis anywhere in the dashboard — no icons, no unicode symbols, no emoji characters
- Deduct 1 point from reviewer satisfaction score if any emojis are found
- Use text labels and CSS styling instead

## TIME DISPLAY — IST TOGGLE
- All timestamps in the dashboard must have an IST toggle (UTC <-> IST)
- Default to IST (UTC+5:30) since the team is in India
- Add a toggle button in the dashboard header to switch between UTC and IST
- Hourly activity charts, session timelines, session table — all must respect this toggle

## MANDATORY: USE OPENAI FOR QUALITATIVE ANALYSIS
- OPENAI_API_KEY is set in .local.env — load it with dotenv
- Use it for qualitative analysis: session arc narratives, prompt effectiveness scoring, intent classification, tacit knowledge extraction, self-correction root cause analysis
- Reference notebooks/003-focused-analytics/003-qualitative-report.py for patterns (LLM calls with disk caching, gpt-4o-mini)
- The reviewer will DEDUCT 2 points if qualitative analysis is still missing after this iteration
- You MUST write a new script (e.g. qualitative_analysis.py) that calls gpt-4o-mini
- Reference: notebooks/003-focused-analytics/003-qualitative-report.py for disk caching pattern
- Output: qualitative_insights.json, session_narratives.json

## COST TRANSPARENCY — NO BUDGET ASSUMPTIONS
- Do NOT say "74% of your entire AI budget" or assume knowledge of the team's budget
- Show raw numbers: total cost, cost per developer, cost per session
- Frame as "your Codex CLI spend is $X/session vs Claude Code at $Y/session" — factual comparison, not budget framing
- All cost calculations must show methodology (e.g. "based on Anthropic API pricing: $X/1M input tokens, $Y/1M output tokens")
- Add an info tooltip on every cost metric explaining how it was calculated
- Show the pricing assumptions used (model, token rates) in an "Assumptions" section or footer

## ANALYSIS ASSUMPTIONS — DISCLOSE EVERYTHING
- Add a dedicated "Methodology & Assumptions" section (collapsible) to the dashboard
- List: date range analyzed, number of sessions, org filter, pricing model used, how session time is calculated, what "active time" means, what "idle" means, how intent classification works, how cost is computed
- If the local DB only has 3-4 days of data, say so prominently — do not let the reader assume this is months of data
- Every insight card should be defensible — if challenged, the methodology must be clear

## CHART HOVER/TOOLTIP THEME — MUST MATCH DARK THEME
- All Recharts tooltip contentStyle must match the dark theme
- Use: background: '#1a1a2e', border: '1px solid rgba(255,255,255,0.1)', borderRadius: 8, color: '#e2e8f0'
- Do NOT use default white/light tooltips — they break the dark theme
- Check ALL chart components for consistent tooltip styling

## NLP PREPROCESSING — STRIP INSTRUCTION TAGS
- Codex CLI messages contain `<INSTRUCTION>...</INSTRUCTION>` system tags that are NOT user prompts
- Before ANY NLP analysis (intent classification, prompt effectiveness, tacit knowledge), strip these tags
- Use regex: re.sub(r'<INSTRUCTION>.*?</INSTRUCTION>', '', text, flags=re.DOTALL)
- Also strip other system tags: `<TOOL_CALL>`, `<SYSTEM>`, etc.
- Apply stopword filtering for word frequency / n-gram analysis
- Failing to strip these will produce garbage NLP results (e.g. "INSTRUCTION" as top keyword)

## QUALITATIVE INSIGHTS MUST BE UNIQUE — NOT REHASHING METRICS
- The AI/qualitative section must provide insights that CANNOT be derived from heuristic analysis
- BAD examples: "Developer X had 50 sessions" (that's just a count), "Codex is more expensive" (that's in cost section already)
- GOOD examples: "Developer X tends to abandon sessions when hitting TypeScript errors — 6 of 8 abandoned sessions involved type mismatches", "The team's prompts become increasingly vague after 4pm IST", "Developer Y's most productive pattern is starting with a specific file path in the prompt"
- The qualitative section should read like a behavioral analyst's report, not a metrics dashboard
- Each finding must cite specific evidence (session IDs, prompt snippets, patterns)
- Deduct 2 points if qualitative insights are just rephrased versions of quantitative metrics

## Status: RUN2_ITER9_ANALYSIS_HOTFIX_COMPLETE
## Iteration: 9
## Phase: Analysis agent fixed stale recommendations (Codex $3.37/session, not $17.03). All reviewer iter9 action items for analysis agent DONE.
## Reviewer Verdict (iter9): KEEP GOING (8.8/10)

## Reviewer Iteration 9 Action Items:
### Analysis Agent:
1. [DONE] Fix stale recommendations — Codex now correctly shows $3.37/session. All 4 recommendations regenerated with current data and new insights (time-of-day, struggle files, prompt length).

### Frontend Agent:
2. [MEDIUM] Add adoption growth context to Hero (sub-text or badge showing 7.7x growth).
3. [LOW] Render per-developer adoption timeline from monthly_trends.developer_adoption.

## Iteration 8 Action Items — ALL DONE:
1. [DONE] Date range fixed. 2. [DONE] Commit text removed. 3. [DONE] Wall hours reconciled. 4. [DONE] Changelog documented.

## Reviewer Iteration 3 Action Items:
### Analysis Agent:
1. [DONE] Strip ALL XML-like tags from all JSON files — catch-all regex, verified 0 remaining with grep.
2. [DONE] Add `cost_per_productive_session` to overview.json — $2.93/productive session vs $1.15/session (2.5x multiplier). 129/327 sessions had edits.
3. [DONE] Add `zero_edit_rate_by_hour` to chart_data.json — 20 hours of data. After 7pm IST, 73-90% zero-edit rate.
4. [DONE] Cap savings_pct at 100% (was 135.1%).
5. [BONUS] Added developer_efficiency ranking (edits per dollar): Vaibhav 5.3, Ajay 2.9, zzjjaayy 1.1 edits/$.

### Frontend Agent:
5. [TODO] Extend useTimezone hook to SessionTimeline.tsx and DailyHeatmap.tsx.
6. [TODO] Add CostInfoTip to DeveloperProfiles.tsx and SessionDeepDives.tsx.
7. [TODO] Display cost_per_productive_session in Hero if available.
8. [TODO] Display zero_edit_rate_by_hour chart if available.

## Iteration 2 Action Items — CLOSED (all addressed):
1-10: All done. active_pct fixed (verified 22.2%), commit keys removed, CostInfoTip added, qualitative rendering complete.

## What was done in Run2 Iter2 (Analysis Agent):
1. **ALL commit fields REMOVED from ALL JSON files** — recursive `remove_commit_keys()` on all 9 files. Text cleanup in developer_summary strings and scatter plot labels.
2. **tacit_knowledge.patterns FIXED** — `string[]` array (30 items) alongside `facts[]`. Frontend can read either.
3. **Developer summaries REGENERATED** — V2 LLM prompt forbids metric restatement. Behavioral patterns with prompt evidence and implications.
4. **active_pct > 100% FIXED** — capped at 100%.
5. **System tags stripped from narratives** — `<environment_context>`, `<INSTRUCTION>` etc. removed. "Warmup"/"clear" replaced with placeholder.
6. **Cost methodology ENHANCED** — per-source pricing (Claude/Codex/Gemini), cached rates, formula, per-dev source costs.
7. **Wall-clock references cleaned** — removed `total_hours_with_ai`, `mean_min`; added `mean_active_min`.
8. **Cost recomputed with correct per-source pricing**.
9. **All 9 JSON files synced** to output/ and dashboard/public/data/.

## What was done in Run2 Iter1 (Analysis Agent):
1. **Session time methodology FIXED** — Now uses sum of inter-message gaps < 30 min (active time), not wall clock. Total active: 130.91h vs 250.39h wall clock (52.3% active rate).
2. **Commit-based metrics REMOVED** — Stripped from overview.json, session_metrics.json, insights.json, cost_by_developer.json, developer_profiles.json.
3. **Cost methodology + assumptions ADDED** — overview.json now has `cost_methodology` and `assumptions` sections with pricing model, formula, data caveats.
4. **Qualitative analysis via gpt-4o-mini COMPLETED**:
   - 20 session narratives (top sessions by activity)
   - 3 developer behavioral summaries (working styles, behavioral patterns, blind spots)
   - 32 tacit knowledge facts + CLAUDE.md suggestions
   - 36 LLM-classified prompt intents (with specificity scores)
   - 10 abandoned session analyses (root cause of zero-edit sessions)
5. **NLP system tags stripped** — INSTRUCTION, TOOL_CALL, SYSTEM tags removed before all text analysis.
6. **No emojis** in any JSON output.
7. **All 9 JSON files synced** to both output/ and dashboard/public/data/.

## New script:
- `analysis-14022026/scripts/run2_iter1_analysis.py` — comprehensive iteration script

## Data validated:
- Local DB has 327 sessions: pratilipi-platform (220) + pratilipi-ai (98)
- Date range: 2025-09-12 to 2026-02-13 (154 days, 5.1 months) — from message timestamps
- Note: first_seen on sessions table is ingestion date, NOT activity date. Always use message timestamps.
- Activity concentrated in Jan-Feb 2026 (230 sessions). Sep-Oct 2025 was early exploration (84 sessions).

## What exists:
- 9 JSON data files in analysis-14022026/output/ AND dashboard/public/data/
- **Full dashboard running on localhost:5173** with 25+ sections across 15 nav tabs
- All JSON files now reflect correct active time methodology
- qualitative_insights.json has rich behavioral analysis (not metric rehash)

## JSON files updated this iteration:
- overview.json — active_time, cost_methodology, assumptions, executive_summary (no commits)
- session_metrics.json — per-session active_min, wall_min fields; commit fields removed
- chart_data.json — active_time dataset rebuilt with proper methodology
- insights.json — commit-based red_flags/insights removed
- cost_by_developer.json — commit fields removed, methodology added
- developer_profiles.json — active_hours per dev, commit fields removed
- qualitative_insights.json — full rebuild: narratives, dev summaries, tacit knowledge, intent classification, abandonment analysis

## New data in qualitative_insights.json:
- `session_narratives` — 20 LLM-generated session arcs with goals, outcomes, difficulty
- `developer_summaries` — 3 behavioral profiles with patterns, blind spots, recommendations
- `tacit_knowledge.facts` — 32 deduped codebase facts the AI keeps rediscovering
- `tacit_knowledge.claude_md_suggestions` — actionable CLAUDE.md entries
- `llm_intent_classification` — 36 prompts classified with intent + specificity score
- `abandonment_analysis` — 10 zero-edit sessions with root cause analysis
- `key_qualitative_findings` — behavioral findings with evidence (not metric rehash)

## What was done in Run2 Iter3 (Frontend Agent):
1. **CRITICAL BUG FIXED — Dashboard black screen**: Hero.tsx crashed on `executive_summary.map()` (was string not array). QualitativeInsights.tsx crashed on `Object.entries(intent_distribution)` (field is `distribution`, not `intent_distribution`). Both fixed with null guards and field fallbacks.
2. **Abandonment analysis rendering fixed**: JSON has dict with `.details[]`, not flat array. Now handles both shapes, plus `reason`/`explanation` field name variants.
3. **CostInfoTip component added**: Reusable `(i)` popover on all cost figures (Hero, CostBreakdown, CostConcentration, RepoCosts) showing full pricing methodology.
4. **CostConcentration default Tooltip fixed**: Stacked bar chart had white background tooltip; now uses mandated dark theme.
5. **All 9 JSON files synced** to dashboard/public/data/.
