# FAQ

Canada provide a wealth of data, open to _everyone_. In most cases, specialised knowledge and tooling is necessary to be able to analyze the datasets. 

This project simplifies that: it enables asking questions in your native language -- the AI will handle the work of translating your questions to answers across a host of data portals.

## How is dataset expiry handled?

Each cached resource has an `expires_at` timestamp computed from the dataset's `update_frequency` field:

| Update frequency | TTL |
|---|---|
| `daily` | 2 days |
| `weekly` | 10 days |
| `monthly` | 45 days |
| `quarterly` | 120 days |
| `biannually` | 200 days |
| `yearly` | 400 days |
| `as_required` / `on_demand` | 365 days |
| Unknown / missing | 30 days |

`expires_at` is written after every download and refresh. Staleness is surfaced passively — in `download_resource` responses, `query_cached` provenance blocks, and `cache_info` output. **No automatic re-download is triggered.** Stale resources stay cached until you call `refresh_cache` or `cache_manage` explicitly.


## Which portals are supported?

Six portals across Ontario:

| Portal | Type |
|---|---|
| Ontario Open Data (`ontario`) | CKAN |
| Toronto Open Data (`toronto`) | CKAN |
| Ottawa Open Data (`ottawa`) | ArcGIS Hub |
| City of Waterloo Open Data (`waterloo`) | ArcGIS Hub |
| City of Kitchener Open Data (`kitchener`) | ArcGIS Hub |
| Region of Waterloo Open Data (`region-waterloo`) | ArcGIS Hub |

All searches fan out to every portal by default. Use the `portal` parameter to narrow to one. Dataset and resource IDs are prefixed with their portal key (e.g. `toronto:abc123`).


## What is DuckDB? Why is it used?

[DuckDB](https://duckdb.org) is an embedded analytical database — fast SQL on local data, no server required. It runs in-process and stores everything in a single file (`~/.cache/ontario-data/ontario_data.duckdb`).

It's used here because:
- Downloaded datasets are materialized directly from pandas DataFrames into DuckDB tables — no intermediate files
- SQL analytics run locally at full speed, avoiding repeated API calls and rate limits
- The file-based design lets multiple MCP server processes share the same cache without lock contention
- The `spatial` extension enables geospatial queries (point-in-polygon, radius searches) natively

Queries are sandboxed: external file and network access is disabled for user-supplied SQL. A 30-second timeout and a 2,000-row result cap apply to all queries.

## How can I reset my local cache?

**Via MCP tools:**
- Remove one resource: `cache_manage` with `action="remove"` and the `resource_id`
- Clear everything: `cache_manage` with `action="clear"`
- Re-download one resource: `refresh_cache` with a `resource_id`
- Re-download all cached resources: `refresh_cache` with no arguments

**Via CLI:**
```sh
ontario-data-mcp cache remove <resource_id>   # remove one
ontario-data-mcp cache clear                   # clear all (prompts for confirmation)
ontario-data-mcp cache clear --yes             # skip confirmation
ontario-data-mcp cache refresh <resource_id>   # re-download one
```

**Manually:** Delete `~/.cache/ontario-data/ontario_data.duckdb`. Override the cache location with the `ONTARIO_DATA_CACHE_DIR` environment variable.


## How large can the local cache get?

There is no size limit or automatic eviction. The cache grows until you clear it or run out of disk space.

Current cache size is visible via:
- `cache_info` MCP tool — returns `total_size_mb` and per-resource `size_mb`
- `ontario-data-mcp cache stats` CLI — prints human-readable total size and row count

Datasets vary widely in size. Manage the cache manually with `cache_manage` or the CLI if space is a concern.
