# Codebase Map

This document is the detailed internal map of `awspipe`.

## Top-Level Layout

- `src/awspipe`: runtime package
- `tests`: non-live test suite
- `docs`: architecture, workflow, and contributor references
- `README.md`: user-facing setup and usage
- `AGENTS.md`: brief repo entrypoint for agents and contributors

## Runtime Modules

### CLI and shell

- `awspipe.cli`
  - defines the top-level Typer app
  - wires `query`, `shell`, `tables`, `inspect`, `doctor`, and `cache clear`
  - constructs the default `QueryEngine`
- `awspipe.repl`
  - interactive shell loop
  - implements dot commands such as `.tables`, `.inspect`, `.schema`, `.connections`, and `.refresh`
- `awspipe.render`
  - output formatting for table, JSON, and CSV render paths

### Query and execution path

- `awspipe.sql`
  - parses SQL with `sqlglot`
  - extracts referenced table names so only needed AWS resources are fetched
- `awspipe.engine`
  - central orchestration layer
  - resolves AWS session context
  - decides whether cached tables are fresh
  - delegates provider fetches and then runs SQL against DuckDB
- `awspipe.backend`
  - owns the DuckDB file, schema bootstrap, table materialization, and metadata tracking
  - exposes `QueryResult`
  - handles cache replacement and full cache clearing

### AWS and provider layer

- `awspipe.aws`
  - resolves profile and region
  - creates the boto3 session
  - calls STS to resolve caller identity and account context
  - powers the `doctor` command
- `awspipe.providers.base`
  - shared provider types: `Column` and `TableProvider`
- `awspipe.providers.aws_tables`
  - built-in AWS table registry
  - provider fetchers for account, region, S3, EC2, VPC, subnet, and security group tables

### Support modules

- `awspipe.paths`
  - resolves cache home and DuckDB file path
  - includes platform-cache fallback behavior for restricted environments
- `awspipe.exceptions`
  - project-specific user-facing exception types
- `awspipe.__main__`
  - module entrypoint for `python -m awspipe`

## Typical Change Paths

### Add a new AWS table

1. Add a provider fetcher and `TableProvider` entry in `awspipe.providers.aws_tables`.
2. Define all public columns, including the shared metadata columns.
3. Make sure the fetcher returns normalized row dictionaries keyed by column name.
4. Add provider tests in `tests/test_providers.py`.
5. Add or adjust engine/backend/CLI tests if the behavior changes.
6. Update `README.md`, `docs/architecture.md`, and `docs/extending-awspipe.md` if the public surface changes.

### Change cache behavior

1. Start in `awspipe.backend` and `awspipe.engine`.
2. Confirm the metadata row shape and freshness logic still line up.
3. Revisit docs that mention cache TTL, partitioning, or `--refresh`.
4. Re-run backend, engine, CLI, and live validation paths.

### Change credential handling

1. Start in `awspipe.aws`.
2. Confirm compatibility with the profile types already documented.
3. Update `README.md`, `AGENTS.md`, `docs/libraries-and-dependencies.md`, and `docs/testing-live-aws.md` if behavior changes.

## Test Coverage Map

- `tests/test_sql.py`: SQL table extraction behavior
- `tests/test_backend.py`: DuckDB materialization and freshness
- `tests/test_engine.py`: query preparation and cache reuse behavior
- `tests/test_aws.py`: session factory and region/profile resolution behavior
- `tests/test_providers.py`: provider normalization and AWS API stubbing
- `tests/test_cli.py`: CLI command surface and output wiring
- `tests/test_paths.py`: cache-path and fallback behavior

## High-Risk Areas

- DuckDB file locking when multiple processes target the same cache path
- AWS `login_session` compatibility via `awscrt`
- provider row shapes drifting from declared columns
- S3 and regional-resource fetch behavior diverging in how `region` is populated
