---
id: m-187c
status: closed
deps: []
links: []
created: 2026-01-27T05:41:59Z
type: task
priority: 2
assignee: chriskd
---
# Add search eval harness

Add a simple evaluation harness that reads a JSON dataset of queries+expected paths and computes Recall@k, MRR, and nDCG@k using mx search. Provide a CLI command to run the evaluation and output summary + per-query results (JSON or table).

## Acceptance Criteria

- [ ] mx eval command loads dataset JSON and runs searches
- [ ] Outputs Recall@k, MRR, nDCG@k, hit rate, and per-query best rank
- [ ] Supports --limit (k), --mode, --scope, --strict, --json
- [ ] Tests cover metrics computations


## Notes

**2026-01-27T05:44:52Z**

Added mx eval command with dataset loader + metrics (recall@k, mrr, ndcg@k, hit rate). Added evaluation metrics tests and verified via pytest and ty check. Manual run with temp dataset.
