# Extending awspipe

Use this document when adding or changing built-in tables.

## Add a New Table

1. Add a fetcher function in `awspipe.providers.aws_tables`.
2. Define a `TableProvider` entry for the new table.
3. Keep the table name consistent with the current SQL naming style, for example `aws_service_resource`.
4. Declare every public column in the provider schema.
5. Return row dictionaries whose keys exactly match the declared column names.

## Required Metadata Columns

Every table currently includes these shared columns:

- `profile_name`
- `account_id`
- `region`
- `arn`
- `title`

New tables should follow that convention unless there is a strong reason not to.

## Fetcher Expectations

- Fetchers receive an `AwsSessionContext`.
- Regional resources should use `context.region` for the service client.
- Fetchers should normalize missing values to `None`.
- Tag-derived titles should prefer the `Name` tag when that pattern makes sense.
- Output should be stable and simple enough to insert directly into DuckDB.

## Cache and Materialization Implications

- `awspipe.engine` decides when a table should be refreshed.
- `awspipe.backend` creates a physical cache table and a public SQL view for each provider.
- Provider schema drift breaks materialization assumptions, so column changes should be deliberate.
- If a table has different freshness or partitioning needs, document that before changing the backend logic.

## Tests to Update

At minimum, review:

- `tests/test_providers.py`
- `tests/test_engine.py`
- `tests/test_cli.py` if discoverability or inspect output changes
- `tests/test_backend.py` if materialization assumptions change

## Docs to Update

Update these when adding or changing a table:

- `README.md` if the user-facing supported table list changes
- `docs/architecture.md` if the provider or cache model changes
- `docs/codebase-map.md` if the edit path changes
- this document if the extension workflow itself changes

## Common Failure Modes

- declared columns do not match row keys
- resource `region` handling is inconsistent with the provider's scope
- AWS paginator or API shape assumptions are wrong
- docs and inspect output drift from the actual provider schema
