# Architecture

`awspipe` has a small runtime surface with a clear split between CLI wiring, query orchestration, AWS session handling, and DuckDB-backed caching.

## Overview

`awspipe` has four runtime layers:

1. CLI and REPL entry points in `awspipe.cli` and `awspipe.repl`
2. Query orchestration in `awspipe.engine`
3. AWS credential/session resolution in `awspipe.aws`
4. DuckDB cache and view materialization in `awspipe.backend`

The provider registry in `awspipe.providers.aws_tables` defines the logical SQL tables exposed by the CLI.

For the detailed source layout and common edit paths, use `docs/codebase-map.md`.

## CLI flow

- `awspipe query` parses SQL, identifies referenced logical tables, refreshes stale cache partitions, and runs the SQL against DuckDB.
- `awspipe shell` reuses the same query engine and adds a small set of Steampipe-style dot commands.
- `awspipe tables` and `awspipe inspect` are registry-driven and do not need live AWS calls.
- `awspipe doctor` validates the AWS CLI credential chain and prints the resolved account, region, and cache path.
- `awspipe cache clear` removes cached DuckDB tables and views, then recreates the metadata schema.

## Provider registry

Each provider declares:

- logical table name
- human description
- stable DuckDB column schema
- scope marker (`global` or `regional`)
- fetcher callable that turns AWS API responses into normalized row dictionaries

The current v1 tables are:

- `aws_account`
- `aws_region`
- `aws_s3_bucket`
- `aws_ec2_instance`
- `aws_vpc`
- `aws_subnet`
- `aws_security_group`

Every table includes these metadata columns:

- `profile_name`
- `account_id`
- `region`
- `arn`
- `title`

For provider implementation details and extension steps, use `docs/extending-awspipe.md`.

## DuckDB cache model

The local cache lives in a single DuckDB file under the platform cache directory or `AWSPIPE_HOME`. If the platform cache directory is not writable, `awspipe` falls back to a local `.awspipe` directory in the current working directory.

For each logical table, `awspipe` creates:

- a physical cache table in the `awspipe_cache` schema named `<table>__data`
- a stable view in the main schema named `<table>`

Each physical table stores two internal partitioning columns:

- `_profile`
- `_source_region`

Cache freshness is tracked in `awspipe_cache.cache_metadata` by logical table, profile, and source region. Query execution refreshes only the referenced tables when the TTL is expired or the user passes `--refresh`.

## Auth and session resolution

Stage 1 uses the normal AWS SDK credential chain through `boto3`.

Resolution rules:

- `--profile` wins when provided
- otherwise `AWS_PROFILE` / shared config and credentials files apply
- region comes from `--region`, then the SDK session, then `AWS_REGION`, then `AWS_DEFAULT_REGION`, then `us-east-1`

The query engine resolves `sts:GetCallerIdentity` once per query run and passes the resulting account context into provider fetchers.

For dependency and profile-type caveats, especially `login_session` support, use `docs/libraries-and-dependencies.md`.

## Future direction

The current code intentionally leaves room for the next step:

- add config-backed named connections
- add assume-role fields per connection
- add aggregator definitions spanning multiple accounts and regions
- isolate more backend-specific logic so SQLite can be added later if needed
