Metadata-Version: 2.4
Name: space-glue
Version: 0.1.1
Summary: SpaCE-GLUE: Spatial Cognition Exercises – General Language Understanding Evaluation
Requires-Python: >=3.13
Description-Content-Type: text/markdown
Requires-Dist: aiohappyeyeballs==2.6.1
Requires-Dist: aiohttp==3.13.2
Requires-Dist: aiosignal==1.4.0
Requires-Dist: annotated-types==0.7.0
Requires-Dist: anyio==4.11.0
Requires-Dist: attrs==25.4.0
Requires-Dist: certifi==2025.11.12
Requires-Dist: charset-normalizer==3.4.4
Requires-Dist: click==8.3.1
Requires-Dist: colorama==0.4.6
Requires-Dist: datasets==3.6.0
Requires-Dist: dill==0.3.8
Requires-Dist: distro==1.9.0
Requires-Dist: dotenv==0.9.9
Requires-Dist: filelock==3.20.0
Requires-Dist: frozenlist==1.8.0
Requires-Dist: fsspec==2025.3.0
Requires-Dist: geographiclib==2.1
Requires-Dist: geopy==2.4.1
Requires-Dist: h11==0.16.0
Requires-Dist: hf-xet==1.2.0
Requires-Dist: httpcore==1.0.9
Requires-Dist: httpx==0.28.1
Requires-Dist: huggingface-hub==1.2.3
Requires-Dist: idna==3.11
Requires-Dist: jiter==0.12.0
Requires-Dist: levenshtein==0.27.3
Requires-Dist: markdown-it-py==4.0.0
Requires-Dist: mdurl==0.1.2
Requires-Dist: mpmath==1.3.0
Requires-Dist: multidict==6.7.0
Requires-Dist: multiprocess==0.70.16
Requires-Dist: numpy==2.3.5
Requires-Dist: openai==2.8.1
Requires-Dist: packaging==25.0
Requires-Dist: pandas==2.3.3
Requires-Dist: propcache==0.4.1
Requires-Dist: pyarrow==22.0.0
Requires-Dist: pydantic==2.12.4
Requires-Dist: pydantic-core==2.41.5
Requires-Dist: pygments==2.19.2
Requires-Dist: python-dateutil==2.9.0.post0
Requires-Dist: python-dotenv==1.2.1
Requires-Dist: python-levenshtein==0.27.3
Requires-Dist: pytz==2025.2
Requires-Dist: pyyaml==6.0.3
Requires-Dist: rapidfuzz==3.14.3
Requires-Dist: requests==2.32.5
Requires-Dist: rich==13.9.4
Requires-Dist: shellingham==1.5.4
Requires-Dist: six==1.17.0
Requires-Dist: sniffio==1.3.1
Requires-Dist: sparc-puzzle==0.3.4
Requires-Dist: sympy==1.14.0
Requires-Dist: tqdm==4.67.1
Requires-Dist: typer-slim==0.20.0
Requires-Dist: typing-extensions==4.15.0
Requires-Dist: typing-inspection==0.4.2
Requires-Dist: tzdata==2025.3
Requires-Dist: urllib3==2.6.2
Requires-Dist: xxhash==3.6.0
Requires-Dist: yarl==1.22.0

# SpaCE-GLUE


## Configuration (`config.yaml`)

This project reads runtime settings from a YAML configuration file. The following
documents the recommended format, the expected data types, which fields are
required, and default values used by the code when a field is omitted.

- **Top-level keys (summary)**
  - `model` (required): Mapping describing the model class and constructor parameters.
  - `datasets` (required, non-empty): Sequence of dataset entries to evaluate.
  - `evaluation` (optional): Settings for evaluation / results handling.
  - `logging` (optional): Logging configuration.

- Any string value set precisely in the form `${VAR_NAME}` will be resolved
from the environment when the config is loaded. The loader calls `load_dotenv()`
so values from a `.env` file are also considered. If the environment variable
is not set, config loading will raise `ValueError`.
- The model´s and each dataset entry's `class` is expected to be a full import path string (e.g. `package.module.ClassName`). The runner will call the configured loader to import and instantiate classes using the `params` mapping.

### `model` (required)

- Type: mapping/dict
- Structure (recommended):

```yaml
model:
  class: "models.openai_model.OpenAIModel"  # string, required: full import path to model class
  params:                                   # mapping of constructor args (optional)
    name: "gpt-4"
    base_url: "https://api.openai.com/v1"
    api_key: "${OPENAI_API_KEY}"
    temperature: 0.7
```


### `datasets` (required, non-empty)

- Type: sequence (list) of mappings
- Each entry (recommended format):

```yaml
datasets:
  - class: "data.example_dataset.ExampleDataset"  # string, required: full import path to dataset class
    params:                                       # mapping of constructor args (optional)
      limit: 20                                    
```


### `evaluation` (optional)

- Type: mapping/dict
- Defaults:
  - `results_dir`: string, default: "results"
  - `batch_size`: int, default: 1
  - `runs`: list[int], default: [1] * len(datasets)
  - `inference`: Bool flag; default `true`.
  - `scoring`: Bool flag; default `true`.

Example:

```yaml
evaluation:
  results_dir: "my_results"
  batch_size: 1
  runs: [3]
  inference: false
  scoring: true
```

### `logging` (optional)

- Type: mapping/dict
- Defaults:
  - `level`: string, default: "INFO" (e.g. "DEBUG", "INFO", "WARNING", "ERROR")
  - `format`: string, default: "%(asctime)s - %(levelname)s - %(message)s"
  - `file`: string, optional (if provided, logs are written to this file)

Example:

```yaml
logging:
  level: "INFO"
  format: "%(asctime)s - %(levelname)s - %(message)s"
  file: SpaCE-GLUE.log
```
