# AGENTS.md — zarr-copy developer guide

## Repository overview

`zarr-copy` is a Python CLI tool that copies and rechunks zarr v3 datasets,
with optional variable renaming/computation via a transform spec.
Two entry points: `zarr-copy` and `zarr-copy-slurm`.

```
zarr_copy/
  __init__.py        # public re-exports
  __main__.py        # zarr-copy CLI
  copy.py            # core copy/rechunk logic
  transform.py       # TransformSpec loading and application
  slurm_rechunk.py   # zarr-copy-slurm CLI
  utils.py           # gen_array, iter_slices, tuple_max
tests/
  test_cli.py
  test_copy.py
  test_slurm.py
  test_transform.py
  test_utils.py
```

---

## Environment

The project uses **uv** for dependency and virtual-environment management.

```bash
uv sync          # create/update .venv and install all deps including dev extras
```

Always run tools via the `.venv` (or `uv run`) so the correct zarr version is used.

---

## Build / lint / test commands

### Run the full test suite
```bash
uv run pytest
# or equivalently:
.venv/bin/python -m pytest
```

Coverage is reported automatically (`--cov=zarr_copy --cov-report=term-missing`
is set in `pyproject.toml`).

### Run a single test file
```bash
uv run pytest tests/test_copy.py -v
```

### Run a single test by name (substring match)
```bash
uv run pytest tests/test_cli.py -v -k "test_multi_source_with_transform"
```

### Run a single test class
```bash
uv run pytest tests/test_transform.py::TestApplyTransform -v
```

### Lint (ruff)
```bash
uv run ruff check zarr_copy tests          # check only
uv run ruff check --fix --unsafe-fixes zarr_copy tests   # auto-fix
uv run ruff format zarr_copy tests         # format
```

### Pre-commit (runs ruff lint + ruff format on staged files)
```bash
pre-commit run --all-files
```

> **Important:** ruff frequently auto-fixes files on the first pre-commit pass.
> If a commit is rejected, run `git add -u && git commit` a second time.

### Build / publish (CI only)
```bash
uv build
uv publish --token $PYPI_TOKEN
```
Publishing requires the `pyproject.toml` `version` field to match the git tag
(`v<major>.<minor>.<patch>`).

---

## Code style

### Formatting
- **Line length:** 100 characters (`tool.ruff` in `pyproject.toml`).
- **Formatter:** `ruff format` (Black-compatible).
- All files must pass `ruff check` with rule sets `E`, `F`, `I`, `UP`.

### Imports
- Always include `from __future__ import annotations` as the first non-docstring
  line in every module — this enables PEP 563 postponed evaluation and is
  present in every source file.
- Import order (enforced by ruff/isort rule `I`): stdlib → third-party → local.
- Use `from __future__ import annotations` before all other imports.
- Prefer `from collections.abc import Callable, Generator, Sequence` over
  `typing.Callable` etc. (ruff UP rules upgrade these automatically).

### Type annotations
- **Target:** Python 3.9+ (`target-version = "py39"` in ruff config).
- Annotate all public function signatures with parameter types and return types.
- Use `X | Y` union syntax (ruff UP rewrites `Union[X, Y]` automatically).
- Use `list[T]`, `dict[K, V]`, `tuple[T, ...]` (lowercase) not `List`, `Dict`.
- `from __future__ import annotations` makes forward references safe everywhere.
- Private helpers (prefix `_`) should still be annotated.

### Naming conventions
- **Functions/variables:** `snake_case`.
- **Classes:** `PascalCase`.
- **Constants/type aliases:** `PascalCase` for type aliases (`TransformSpec`),
  `UPPER_SNAKE` for true constants (none currently).
- **Private helpers:** leading underscore, e.g. `_find_var`, `_find_primary`,
  `_entry_to_source`, `_full_slices`.
- **CLI entry points** (`main`) are module-level functions, not classes.

### Docstrings
- Every public function has a Google/NumPy-style docstring with a one-line
  summary, then `Parameters`, `Returns`, and `Raises` sections as needed.
- Private helpers have a brief one-line docstring.
- Modules have a one-line module docstring at the top.

### Error handling
- Raise `KeyError` for missing source variables (callers can catch it at the
  CLI boundary and print a human-readable message).
- Raise `ValueError` for invalid configuration (bad transform spec, bad
  callable, etc.).
- Raise `FileNotFoundError` for missing files (e.g. transform config).
- In `main()` functions: catch exceptions at the top level, print to
  `sys.stderr`, and return a non-zero int — **do not call `parser.error()`**
  (it raises `SystemExit`, which breaks tests that check the return code).
  Use `print("...", file=sys.stderr); return 1` instead.

---

## zarr v3 API notes

The project pins **zarr ≥ 2.18** but the code targets the zarr v3 API:

| Task | Correct v3 call |
|------|-----------------|
| Create array | `group.create_array(name, shape=..., chunks=..., dtype=..., compressors=..., overwrite=True)` |
| Set attributes | `array.update_attributes({...})` |
| Blosc codec | `from zarr.codecs import BloscCodec; BloscCodec(cname="lz4", clevel=1, shuffle="noshuffle")` |
| Open group | `zarr.open_group(path, mode="r"|"a"|"w")` |

- `LocalStore` has **no `.path` attribute** in zarr v3 — use the string path
  directly (e.g. in tests: `str(tmp_path / "src.zarr")`, not
  `grp.store.path`).
- `array.compressors` is the correct attribute (not `array.compressor`).

---

## Testing conventions

- Test files mirror source modules: `zarr_copy/copy.py` → `tests/test_copy.py`.
- Tests are grouped into classes: `TestCopyVariable`, `TestCopyGroup`, etc.
- Use `pytest.fixture` with `tmp_path` for all zarr stores; never use fixed
  paths on disk.
- Fixtures that create zarr stores return the **string path** (not the group
  object) when they are consumed by CLI tests, so `main([path, dst])` works.
- Fixtures that create zarr stores return the **group object** for unit tests
  on copy/transform functions.
- Use `np.testing.assert_array_equal` / `np.testing.assert_allclose` for data
  assertions, not plain `==`.
- Assert return codes: `assert rc == 0` or `assert rc != 0` — avoid
  `pytest.raises(SystemExit)` for CLI tests.

---

## Transform spec

A transform spec is a Python file containing a module-level `transform` dict:

```python
# transform.py
transform = {
    "rlut": lambda ttr: ttr / -3600,   # callable: param names = source var names
    "psl":  "msl",                     # string: simple rename/copy
}
```

- `load_transform_spec(path)` — import and validate the file.
- `resolve_sources(func)` — extract parameter names from a callable.
- `apply_transform(region, entry, src_groups, dst_var, dst_name)` — write one
  chunk region.
- `copy_variable_transformed(dst_name, entry, src_groups, dst_group, ...)` —
  full variable copy with transform.

`src_groups` accepts either a single `zarr.Group` or `list[zarr.Group]`;
source variables are searched across all groups in order (first match wins).

---

## CLI positional argument convention

Both `zarr-copy` and `zarr-copy-slurm` use `zarr_paths` (`nargs="+"`) where
**the last argument is always the destination**:

```
zarr-copy SRC [SRC ...] DST [options]
```

Multiple sources are only allowed when `--transform` is provided.
`build_job_script` takes `src_paths: list[str]` (not `zarr_in: str`).
