# Typecheck Testdata Management Policy

## Overview

- This directory contains example subsets of autogenerated testdata for ABI and codec overload type checking.
- The full testdata set is not stored in git; only the first 5 files per subfolder are kept as examples.
- The full set can be generated on demand using `scripts/generate_overload_tests.py`.

## Maintenance Instructions

- **To regenerate testdata:**
  Run:

  ```bash
  # Generate all testdata for both ABI and codec, all lengths
  python scripts/generate_overload_tests.py --impl both

  # Generate only a specific length (e.g., length 3) for both ABI and codec
  python scripts/generate_overload_tests.py --impl both --lengths 3

  # Generate only a random 1/10th of the test files for length 3 (useful for quick checks)
  python scripts/generate_overload_tests.py --impl both --lengths 3 --sample-1-of-x 10

  # Generate only a random 1/200th of the test files for length 4, with a fixed seed for reproducibility
  python scripts/generate_overload_tests.py --impl both --lengths 4 --sample-1-of-x 200 --seed 42
  ```

  This will overwrite all testdata files in `abi/` and `codec/` subdirectories.

- **.gitignore policy:**
  Each subfolder should have a .gitignore that keeps only the first 5 files:

  ```
  overload_test_data_*.py
  !overload_test_data_00001.py
  !overload_test_data_00002.py
  !overload_test_data_00003.py
  !overload_test_data_00004.py
  !overload_test_data_00005.py
  ```

- **When to update testdata:**

  - When ABI type string definitions change.
  - When the overload test generation script is updated.
  - When adding new test coverage for type checking.

- **CI Integration:**

  - The GitHub Actions workflow `.github/workflows/mypy.yaml` uses these flags to efficiently generate and check a representative subset of testdata for large lengths.
  - See the workflow file for details on how sampling and length selection are used in CI.
  - Ensure the testdata is present or generated before running CI.

## For maintainers

- Do not manually edit testdata files; always use the script.
- Update this policy if the testdata management approach changes.
- Keep this directory in sync with the latest ABI and codec overload definitions.
