# Test Plan

## Bench Orchestration (`tests/test_bench.py`)

| Test | Behavior | Code paths |
| --- | --- | --- |
| `test_cmd_download_with_stubs` | Replaces `load_composite` and `export_images` so `cmd_download` runs without touching external services, confirming it honors `size` routing, ensures `ensure_dir` succeeds, and saves the dataset through the stubbed object. | `src/inatinqperf/bench/bench.py:71`, `src/inatinqperf/bench/bench.py:73`, `src/inatinqperf/bench/bench.py:79`, `src/inatinqperf/bench/bench.py:81`, `src/inatinqperf/bench/bench.py:85` |
| `test_cmd_download_no_export` | Forces the CLI override `--export-images` to `False` and verifies the download command skips `export_images`, proving argument-level flags take precedence over config defaults. | `src/inatinqperf/bench/bench.py:74`, `src/inatinqperf/bench/bench.py:75`, `src/inatinqperf/bench/bench.py:83` |
| `test_cmd_embed_with_stubs` | Stubs `embed_images` and `to_hf_dataset` to exercise `cmd_embed`’s default-config path, proving it creates output directories, propagates embeddings, and calls `save_to_disk`. | `src/inatinqperf/bench/bench.py:92`, `src/inatinqperf/bench/bench.py:97`, `src/inatinqperf/bench/bench.py:100`, `src/inatinqperf/bench/bench.py:101`, `src/inatinqperf/bench/bench.py:102` |
| `test_cmd_embed_with_overrides` | Supplies all CLI arguments so `cmd_embed` reads from arguments instead of config, validating the override logic while reusing stubs for the heavy work. | `src/inatinqperf/bench/bench.py:92`, `src/inatinqperf/bench/bench.py:93`, `src/inatinqperf/bench/bench.py:94`, `src/inatinqperf/bench/bench.py:95`, `src/inatinqperf/bench/bench.py:100` |
| `test_cmd_build_with_dummy_backend` | Replaces dataset loading with synthetic embeddings to confirm `cmd_build` stacks vectors, instantiates the backend via `_init_backend`, calls `train`/`upsert`, and prints stats. | `src/inatinqperf/bench/bench.py:123`, `src/inatinqperf/bench/bench.py:126`, `src/inatinqperf/bench/bench.py:127`, `src/inatinqperf/bench/bench.py:129`, `src/inatinqperf/bench/bench.py:137`, `src/inatinqperf/bench/bench.py:138` |
| `test_cmd_search_safe_pickle_and_backend` | Swaps FAISS objects with safe stubs so `cmd_search` builds both exact and approximate indexes, embeds text queries, computes recall, and prints the JSON summary without touching real FAISS code. | `src/inatinqperf/bench/bench.py:145`, `src/inatinqperf/bench/bench.py:151`, `src/inatinqperf/bench/bench.py:156`, `src/inatinqperf/bench/bench.py:158`, `src/inatinqperf/bench/bench.py:166`, `src/inatinqperf/bench/bench.py:170`, `src/inatinqperf/bench/bench.py:175`, `src/inatinqperf/bench/bench.py:181`, `src/inatinqperf/bench/bench.py:188` |
| `test_cmd_update_with_dummy_backend` | Uses fake embeddings to drive `cmd_update` through training, initial `upsert`, synthetic add/delete vectors, and the final stats print, ensuring the update workflow sequences correctly. | `src/inatinqperf/bench/bench.py:197`, `src/inatinqperf/bench/bench.py:202`, `src/inatinqperf/bench/bench.py:205`, `src/inatinqperf/bench/bench.py:206`, `src/inatinqperf/bench/bench.py:217`, `src/inatinqperf/bench/bench.py:221`, `src/inatinqperf/bench/bench.py:223` |
| `test_cli_main_dispatch` | Monkeypatches each subcommand and mutates `sys.argv` for every verb, proving the CLI parser wires each subcommand to the right handler after loading the default config. | `src/inatinqperf/bench/bench.py:270`, `src/inatinqperf/bench/bench.py:273`, `src/inatinqperf/bench/bench.py:280`, `src/inatinqperf/bench/bench.py:287`, `src/inatinqperf/bench/bench.py:292`, `src/inatinqperf/bench/bench.py:299`, `src/inatinqperf/bench/bench.py:306`, `src/inatinqperf/bench/bench.py:313`, `src/inatinqperf/bench/bench.py:315` |
| `test_cmd_run_all_calls_sequence` | Replaces every stage with a recorder stub and asserts `cmd_run_all` invokes download → embed → build (baseline first, then chosen backend) → search → update in order. | `src/inatinqperf/bench/bench.py:226`, `src/inatinqperf/bench/bench.py:233`, `src/inatinqperf/bench/bench.py:237`, `src/inatinqperf/bench/bench.py:244`, `src/inatinqperf/bench/bench.py:247`, `src/inatinqperf/bench/bench.py:251`, `src/inatinqperf/bench/bench.py:258`, `src/inatinqperf/bench/bench.py:265` |
| `test_recall_at_k_edges` | Exercises `recall_at_k` with empty and truncated neighbor arrays to confirm safe handling of degenerate cases and bounds checking on `k`. | `src/inatinqperf/bench/bench.py:61`, `src/inatinqperf/bench/bench.py:64`, `src/inatinqperf/bench/bench.py:66` |
| `test_load_cfg_and_ensure_dir_error_and_idempotency` | Confirms YAML config loading works, directory creation is idempotent, and missing files raise the expected `FileNotFoundError`/`OSError`. | `src/inatinqperf/bench/bench.py:38`, `src/inatinqperf/bench/bench.py:40`, `src/inatinqperf/bench/bench.py:45`, `src/inatinqperf/bench/bench.py:47` |
| `test_init_backend_scrubs_reserved` | Injects a capture backend into `ALL_BACKENDS` and verifies `_init_backend` removes reserved keys (`metric`, `dim`, `name`) before calling `init`, preventing duplicate-argument errors. | `src/inatinqperf/bench/bench.py:111`, `src/inatinqperf/bench/bench.py:112`, `src/inatinqperf/bench/bench.py:114`, `src/inatinqperf/bench/bench.py:116`, `src/inatinqperf/bench/bench.py:117` |

## Data I/O Utilities (`tests/test_dataio.py`)

| Test | Behavior | Code paths |
| --- | --- | --- |
| `test_load_composite_all_parts_fail_falls_back_to_train` | Simulates split failures so `load_composite` retries additional splits, eventually falls back to `train`, and logs the warning message. | `src/inatinqperf/utils/dataio.py:12`, `src/inatinqperf/utils/dataio.py:16`, `src/inatinqperf/utils/dataio.py:18`, `src/inatinqperf/utils/dataio.py:20` |
| `test_load_composite_single_part_avoids_concat` | Ensures a single-split request returns directly without calling `concatenate_datasets`, guarding against unnecessary concatenation work. | `src/inatinqperf/utils/dataio.py:21`, `src/inatinqperf/utils/dataio.py:22` |
| `test_load_composite_multiple_parts_concatenates` | Confirms multi-split requests call the patched `concatenate_datasets`, yielding a combined dataset marked with a sentinel attribute. | `src/inatinqperf/utils/dataio.py:12`, `src/inatinqperf/utils/dataio.py:22`, `src/inatinqperf/utils/dataio.py:23` |
| `test_export_images_writes_jpegs_and_manifest` | Runs `export_images` with mixed PIL/NumPy inputs, verifying JPEG emission, manifest contents, label serialization, and image dimensions on disk. | `src/inatinqperf/utils/dataio.py:27`, `src/inatinqperf/utils/dataio.py:31`, `src/inatinqperf/utils/dataio.py:34`, `src/inatinqperf/utils/dataio.py:35`, `src/inatinqperf/utils/dataio.py:36`, `src/inatinqperf/utils/dataio.py:40` |

## Embedding Utilities (`tests/test_embed.py`)

| Test | Behavior | Code paths |
| --- | --- | --- |
| `test_pilify_numpy_and_pil` | Validates `pilify` converts NumPy arrays to RGB PIL images and passes existing PIL objects through unchanged. | `src/inatinqperf/utils/embed.py:15`, `src/inatinqperf/utils/embed.py:18`, `src/inatinqperf/utils/embed.py:20` |
| `test_pilify_invalid_type` | Confirms `pilify` rejects unsupported types by raising `TypeError`. | `src/inatinqperf/utils/embed.py:21`, `src/inatinqperf/utils/embed.py:22` |
| `test_embed_images_empty_dataset` | Stubs dataset/model/processor constructors to prove `embed_images` handles an empty HF dataset gracefully and returns empty embeddings/metadata. | `src/inatinqperf/utils/embed.py:27`, `src/inatinqperf/utils/embed.py:33`, `src/inatinqperf/utils/embed.py:48`, `src/inatinqperf/utils/embed.py:56` |
| `test_embed_images_model_error` | Forces `CLIPModel.get_image_features` to raise, ensuring exceptions surface to callers instead of being swallowed. | `src/inatinqperf/utils/embed.py:37`, `src/inatinqperf/utils/embed.py:38`, `src/inatinqperf/utils/embed.py:39` |
| `test_to_hf_dataset_structure` | Verifies `to_hf_dataset` builds a dataset with `id`, `label`, and list-backed `embedding` columns containing the supplied data. | `src/inatinqperf/utils/embed.py:65`, `src/inatinqperf/utils/embed.py:69`, `src/inatinqperf/utils/embed.py:75`, `src/inatinqperf/utils/embed.py:82`, `src/inatinqperf/utils/embed.py:85` |
| `test_embed_images_and_to_hf_dataset` | Runs `embed_images` end-to-end on a fake dataset to confirm batching, ID/label extraction, and shape of the returned NumPy array before converting to a Hugging Face dataset. | `src/inatinqperf/utils/embed.py:33`, `src/inatinqperf/utils/embed.py:34`, `src/inatinqperf/utils/embed.py:37`, `src/inatinqperf/utils/embed.py:41`, `src/inatinqperf/utils/embed.py:42`, `src/inatinqperf/utils/embed.py:75`, `src/inatinqperf/utils/embed.py:82` |
| `test_embed_text` | Ensures `embed_text` returns a NumPy array with one row per query when provided with dummy processor/model stubs. | `src/inatinqperf/utils/embed.py:94`, `src/inatinqperf/utils/embed.py:98`, `src/inatinqperf/utils/embed.py:100`, `src/inatinqperf/utils/embed.py:101` |
| `test_embed_text_empty` | Confirms `embed_text` short-circuits to an empty array when given no queries. | `src/inatinqperf/utils/embed.py:94`, `src/inatinqperf/utils/embed.py:98`, `src/inatinqperf/utils/embed.py:101` |

## Profiler (`tests/test_profiler.py`)

| Test | Behavior | Code paths |
| --- | --- | --- |
| `test_profiler_writes_metrics_and_json` | Uses a temporary results directory to ensure `Profiler` captures timing/memory metrics, exposes them via `metrics`, and writes a JSON report to disk. | `src/inatinqperf/utils/profiler.py:23`, `src/inatinqperf/utils/profiler.py:31`, `src/inatinqperf/utils/profiler.py:38`, `src/inatinqperf/utils/profiler.py:51`, `src/inatinqperf/utils/profiler.py:59`, `src/inatinqperf/utils/profiler.py:70` |
| `test_profiler_sample_handles_exceptions` | Patches `memory_info` to raise and verifies `Profiler.sample` swallows the error so profiling can continue. | `src/inatinqperf/utils/profiler.py:38`, `src/inatinqperf/utils/profiler.py:41`, `src/inatinqperf/utils/profiler.py:43` |
| `test_profiler_multiple_steps_create_distinct_files` | Runs two profiler instances and asserts each writes a unique JSON file with distinct step names, demonstrating isolated contexts. | `src/inatinqperf/utils/profiler.py:23`, `src/inatinqperf/utils/profiler.py:31`, `src/inatinqperf/utils/profiler.py:70` |

## Vector Backend Base Class (`tests/test_vector_backend.py`)

| Test | Behavior | Code paths |
| --- | --- | --- |
| `test_vectorbackend_is_abstract` | Asserts the abstract `VectorBackend` cannot be instantiated directly, enforcing the ABC contract. | `src/inatinqperf/adaptors/base.py:9`, `src/inatinqperf/adaptors/base.py:14`, `src/inatinqperf/adaptors/base.py:18`, `src/inatinqperf/adaptors/base.py:23`, `src/inatinqperf/adaptors/base.py:27`, `src/inatinqperf/adaptors/base.py:32`, `src/inatinqperf/adaptors/base.py:37`, `src/inatinqperf/adaptors/base.py:42` |
| `test_partial_implementation_rejected` | Defines a subclass missing required methods and confirms instantiation still raises, ensuring the abstract interface is enforced. | `src/inatinqperf/adaptors/base.py:14`, `src/inatinqperf/adaptors/base.py:18`, `src/inatinqperf/adaptors/base.py:23`, `src/inatinqperf/adaptors/base.py:27`, `src/inatinqperf/adaptors/base.py:32`, `src/inatinqperf/adaptors/base.py:37`, `src/inatinqperf/adaptors/base.py:42` |
| `test_lifecycle_and_shapes` | Implements a concrete `DummyBackend` and walks through `init` → `train` → `upsert` → `search` → `delete` → `drop`, asserting shape/dtype correctness and stats tracking for both `ip` and `l2` metrics. | `src/inatinqperf/adaptors/base.py:14`, `src/inatinqperf/adaptors/base.py:18`, `src/inatinqperf/adaptors/base.py:23`, `src/inatinqperf/adaptors/base.py:27`, `src/inatinqperf/adaptors/base.py:32`, `src/inatinqperf/adaptors/base.py:37`, `src/inatinqperf/adaptors/base.py:42` |
| `test_upsert_replaces_existing` | Verifies `DummyBackend.upsert` overwrites existing IDs instead of duplicating them and that subsequent searches use the updated vectors. | `src/inatinqperf/adaptors/base.py:14`, `src/inatinqperf/adaptors/base.py:23`, `src/inatinqperf/adaptors/base.py:27` |

## FAISS Backends (`tests/test_faiss_backend.py`)

| Test | Behavior | Code paths |
| --- | --- | --- |
| `test_faiss_flat_lifecycle` | Covers `FaissFlat` initialization for both metrics, insertion, search shape guarantees, delete semantics, and idempotent `drop`. | `src/inatinqperf/adaptors/faiss_backend.py:26`, `src/inatinqperf/adaptors/faiss_backend.py:30`, `src/inatinqperf/adaptors/faiss_backend.py:36`, `src/inatinqperf/adaptors/faiss_backend.py:41`, `src/inatinqperf/adaptors/faiss_backend.py:46`, `src/inatinqperf/adaptors/faiss_backend.py:55` |
| `test_faiss_ivfpq_train_and_search_with_large_training` | Trains `FaissIVFPQ` on a large synthetic set to avoid FAISS warnings, checks stats, runs deterministic searches, and validates delete/drop behavior. | `src/inatinqperf/adaptors/faiss_backend.py:100`, `src/inatinqperf/adaptors/faiss_backend.py:104`, `src/inatinqperf/adaptors/faiss_backend.py:115`, `src/inatinqperf/adaptors/faiss_backend.py:121`, `src/inatinqperf/adaptors/faiss_backend.py:138`, `src/inatinqperf/adaptors/faiss_backend.py:145`, `src/inatinqperf/adaptors/faiss_backend.py:148`, `src/inatinqperf/adaptors/faiss_backend.py:158`, `src/inatinqperf/adaptors/faiss_backend.py:167`, `src/inatinqperf/adaptors/faiss_backend.py:178` |
| `test_faiss_ivfpq_l2_metric_delete_and_drop` | Exercises the L2 metric path, runtime `nprobe` override during search, deletion of missing/existing IDs, and repeated `drop` calls. | `src/inatinqperf/adaptors/faiss_backend.py:100`, `src/inatinqperf/adaptors/faiss_backend.py:115`, `src/inatinqperf/adaptors/faiss_backend.py:148`, `src/inatinqperf/adaptors/faiss_backend.py:153`, `src/inatinqperf/adaptors/faiss_backend.py:158`, `src/inatinqperf/adaptors/faiss_backend.py:164`, `src/inatinqperf/adaptors/faiss_backend.py:178` |
| `test_faiss_ivfpq_runtime_nprobe_override_and_upsert_replace` | Ensures IVFPQ upsert replaces vectors in-place and that the `nprobe` argument at search time overrides the configured value. | `src/inatinqperf/adaptors/faiss_backend.py:100`, `src/inatinqperf/adaptors/faiss_backend.py:115`, `src/inatinqperf/adaptors/faiss_backend.py:148`, `src/inatinqperf/adaptors/faiss_backend.py:158`, `src/inatinqperf/adaptors/faiss_backend.py:164` |
| `test_faiss_flat_topk_greater_than_ntotal_and_idempotent_delete` | Requests more neighbors than stored to ensure padding is handled, and verifies deleting unknown IDs is a no-op. | `src/inatinqperf/adaptors/faiss_backend.py:36`, `src/inatinqperf/adaptors/faiss_backend.py:41`, `src/inatinqperf/adaptors/faiss_backend.py:46`, `src/inatinqperf/adaptors/faiss_backend.py:49`, `src/inatinqperf/adaptors/faiss_backend.py:55` |
| `test_metric_mapping_and_unwrap_real_index` | Exercises `_metric_to_faiss` fallbacks and `_unwrap_to_ivf` against a real FAISS index to guarantee the helper can penetrate wrapper layers. | `src/inatinqperf/adaptors/faiss_backend.py:10`, `src/inatinqperf/adaptors/faiss_backend.py:12`, `src/inatinqperf/adaptors/faiss_backend.py:60`, `src/inatinqperf/adaptors/faiss_backend.py:68`, `src/inatinqperf/adaptors/faiss_backend.py:74`, `src/inatinqperf/adaptors/faiss_backend.py:79` |
| `test_faiss_ivfpq_cosine_metric_and_topk_gt_ntotal` | Validates cosine-metric initialization, large `topk` searches (with `-1` padding), and clean resource release. | `src/inatinqperf/adaptors/faiss_backend.py:100`, `src/inatinqperf/adaptors/faiss_backend.py:111`, `src/inatinqperf/adaptors/faiss_backend.py:148`, `src/inatinqperf/adaptors/faiss_backend.py:158`, `src/inatinqperf/adaptors/faiss_backend.py:164`, `src/inatinqperf/adaptors/faiss_backend.py:178` |
| `test_faiss_ivfpq_empty_delete_noop` | Confirms attempting to delete an empty list leaves the index unchanged. | `src/inatinqperf/adaptors/faiss_backend.py:148`, `src/inatinqperf/adaptors/faiss_backend.py:153`, `src/inatinqperf/adaptors/faiss_backend.py:155` |
| `test_unwrap_fallback_dummy_chain` | Builds a manual wrapper chain to cover the fallback logic of `_unwrap_to_ivf` when `faiss.extract_index_ivf` is unavailable. | `src/inatinqperf/adaptors/faiss_backend.py:74`, `src/inatinqperf/adaptors/faiss_backend.py:78`, `src/inatinqperf/adaptors/faiss_backend.py:79`, `src/inatinqperf/adaptors/faiss_backend.py:82`, `src/inatinqperf/adaptors/faiss_backend.py:83` |
| `test_metric_to_faiss_ip_mapping` | Explicitly checks the `'ip'` branch of `_metric_to_faiss` to guarantee consistent mapping. | `src/inatinqperf/adaptors/faiss_backend.py:10`, `src/inatinqperf/adaptors/faiss_backend.py:11` |

## Scaffolding Sample (`tests/UvScaffolding_sample_file_test.py`)

| Test | Behavior | Code paths |
| --- | --- | --- |
| `test_Add` and `test_Subtract` | Smoke-test the autogenerated arithmetic helpers to ensure the template wiring is intact. | `src/inatinqperf/UvScaffolding_sample_file.py:4`, `src/inatinqperf/UvScaffolding_sample_file.py:9` |
| `test_hello` | Verifies the package-level `hello` function returns the expected greeting string. | `src/inatinqperf/__init__.py:4`, `src/inatinqperf/__init__.py:6` |
