Metadata-Version: 2.1
Name: model_config_tests
Version: 0.0.1
Summary: Test for ACCESS model (payu) configurations
Author: ACCESS-NRI
License: Apache-2.0
Project-URL: Homepage, https://github.com/ACCESS-NRI/model-config-tests/
Project-URL: Issues, https://github.com/ACCESS-NRI/model-config-tests/issues
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: POSIX :: Linux
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: f90nml >=0.16
Requires-Dist: requests
Requires-Dist: PyYAML
Requires-Dist: pytest >=8.0.1
Requires-Dist: ruamel.yaml >=0.18.5
Requires-Dist: jsonschema >=4.21.1
Requires-Dist: payu >=1.1.3
Provides-Extra: test
Requires-Dist: pytest-cov ; extra == 'test'

## Model Configuration Pytests

These pytests are used as part CI checks for model configurations.

The checksum pytests are used for reproducibility CI checks in the [ACCESS-NRI/reproducibility](https://github.com/ACCESS-NRI/reproducibility) repository. The quick configuration tests are used in 
[ACCESS-NRI/access-om2-configs](https://github.com/ACCESS-NRI/access-om2-configs).

Code from these tests is adapted from COSIMAS's ACCESS-OM2's [
bit reproducibility tests](https://github.com/COSIMA/access-om2/blob/master/test/test_bit_reproducibility.py).

### How to run tests manually (from a local install of model-config-tests)

1. First clone the pytest code into a separate directory.
```sh
git clone https://github.com/ACCESS-NRI/model-config-tests/ model-config-tests
```

2. To create a local install of `model-config-tests`
```sh
cd model-config-tests
pip install --user -e .
model-config-tests --help # model-config-tests runs the pytest command for tests in the package
```

2. Checkout an experiment (in this case it is using an ACCESS-OM2 config)
```sh
git clone https://github.com/ACCESS-NRI/access-om2-configs/ <experiment>
cd <experiment>
git checkout <branch/tag>
```

3. Setup payu
```sh
module use /g/data/vk83/modules
module load payu/1.1.3
```

4. Run the pytests
```sh
model-config-tests
```

### Pytest Options

The output directory for pytests defaults to `$TMPDIR/test-model-repro` and contains the following subdirectories:
- `control` - contains copies of the model configuration used for each experiment run in the tests.
- `lab` - contains `payu` model output directories containing `work` and `archive` sub-directories.

This output directory also contains files generated by pytests, including the `CHECKSUM` file which is used as part of reproducibility CI workflows.

To specify a different folder for pytest outputs, use `--output-path` command flag, for example:

```sh
model-config-tests --output-path /some/other/path/for/output
```

By default, the control directory, e.g. the model configuration to test, is the current working directory. This can be set similarly to above by using the 
`--control-path` command flag.

The path containing the checksum file to check against can also be set using
`--checksum-path` command flag. The default is the `testing/checksum/CHECKSUM`
file which is stored in the control directory.

To run only CI reproducibility checksum tests, use `-m checksum`, e.g.

```sh
model-config-tests -m checksum
```

To run quick configuration tests, use the `config` marker. To additionally run
ACCESS-OM2 specific quick configuration tests, use `access_om2` marker,
e.g.:

```sh
model-config-tests -m "config or access_om2"
```
