# dattos

CLI for the [Dattos](https://dattos.com.br) reconciliation platform — designed as an **agent-native interface** for AI agents (Claude Code, custom agents) to operate Dattos programmatically.

All output is **JSON to stdout**, errors go to **stderr**, making it trivially parseable by both humans and machines.

## Installation

Requires **Python 3.11+**.

```bash
pip install dattos

# Verify
dattos version
# {"version": "1.0.0"}
```

### From source (development)

```bash
git clone git@bitbucket.org:dattos/dattos-cli.git
cd dattos-cli
pip install -e .
```

## Authentication

The CLI supports **API keys** and **interactive login** (session tokens). Credentials are stored securely in your OS keyring (Windows Credential Manager / macOS Keychain / Linux Secret Service).

### Quick Setup

```bash
dattos config set api-url "https://your-instance.dattos.com.br/dattos.api"
dattos config set api-key "api-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
```

### Interactive Login

```bash
# Log in with username and password (stores session token in keyring)
dattos auth login

# Check current auth method
dattos auth status
# {"method": "session", "profile": "default", "token": "abc12345***"}

# Log out (invalidates session)
dattos auth logout
```

### Environment Variables

```bash
export DATTOS_API_URL="https://your-instance.dattos.com.br/dattos.api"
export DATTOS_API_KEY="api-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
export DATTOS_FOLDER_ID="2473"  # optional, default folder for all commands
```

Environment variables take precedence over keyring and config file.

### Folder Selection

Most Dattos API endpoints require a **folder context**. Use `dattos folders select` to pick a folder interactively or by ID:

```bash
# Interactive picker (TTY only — shows numbered list)
dattos folders select

# Direct selection by ID
dattos folders select 2473

# Check which folder is active
dattos folders current
# {"folder_id": "2473", "folder_name": "sirio"}
```

You can also discover folders first:

```bash
# Full tree (raw JSON)
dattos folders list

# Flat list with id, name, and full path (easier to read)
dattos folders list --flat
```

### Profiles

Manage multiple environments with named profiles.

```bash
# Create and switch to a staging profile
dattos config profile switch staging
dattos config set api-url "https://staging.dattos.com.br/dattos.api"
dattos config set api-key "api-staging-key"

# Switch back to default
dattos config profile switch default

# Use a profile for a single command
dattos --profile staging analyses list

# Or via environment variable
DATTOS_PROFILE=staging dattos analyses list

# List all profiles
dattos config profile list
```

### Shell Completion

The CLI supports tab completion for Bash, Zsh, Fish, and PowerShell.

```bash
# Install completion for your shell
dattos --install-completion

# Or view the completion script without installing
dattos --show-completion
```

After installing, restart your shell or source the completion file.

## Commands

### `dattos analyses` — Manage analyses

```bash
# List analyses (with pagination and search)
dattos analyses list --page-size 10
# {"data": [{"analysisId": 5192, "analysisCode": "PRD-7728", "analysisName": "Reconciliação Bancária", ...}], "totalItems": 42}

dattos analyses list --search "RECON-001" --folder-id 2473
dattos analyses list --inactive  # include inactive/archived

# Get analysis by ID
dattos analyses get 5192 --folder-id 2473
# {"analysisId": 5192, "analysisCode": "PRD-7728", "analysisName": "Reconciliação Bancária", "isActive": true, ...}

# Get analysis by code
dattos analyses get-by-code "ETL-1" --folder-id 2473
# {"analysisId": 4137, "analysisCode": "ETL-1", ...}

# List data sources and connector info
dattos analyses datasources 5192 --folder-id 2473
# [{"id": 101, "name": "Extrato Bancário", "connectorType": "Excel", ...}]

# Check execution status at a reference date
dattos analyses execution-status 5192 --date 2025-12-20 --folder-id 2473
# {"executionJobId": "89625", "status": "Loaded", "completedPercent": 100.0, ...}
# Status values: NoLoad, Loading, Loaded, Error, Inactive, InvalidPreparation

# Start an analysis execution (default: replace mode)
dattos analyses start 5192 --date 2026-03-08
# {"status": "started", "analysisId": 5192, "referenceDate": "2026-03-08"}

# Start with file upload (auto-uploads, then starts)
dattos analyses start 5192 --date 2026-03-08 --file data.csv
# Supported: .csv, .edi, .ofx, .pdf, .rem, .ret, .txt, .xlsx, .xls, .xlsb, .xml

# Start and wait for completion (polls every 5s)
dattos analyses start 5192 --date 2026-03-08 --wait

# Incremental import (preserves existing data, adds new)
dattos analyses start 5192 --date 2026-03-08 --incremental
```

### `dattos loads` — Query analysis loads

```bash
# Get load details for a reference date (returns load ID + dataset IDs)
dattos loads get 5192 2025-12-20 --folder-id 2473
# {"id": 3248, "referenceDate": "2025-12-20", "datasets": [{"id": 2476, "name": "Extrato", "columns": [...]}]}

# List all reference dates with loaded data
dattos loads list-dates 4137 --from 2025-01-01 --to 2026-12-31 --folder-id 2473
# ["2025-12-09T00:00:00", "2025-12-10T00:00:00", "2025-12-11T00:00:00"]

# List available views for a load
dattos loads views 5192 3248 --folder-id 2473
# [{"id": 1, "name": "Summary"}, {"id": 2, "name": "Detail"}]
```

### `dattos datasets` — Query dataset rows

```bash
# Search rows in a dataset (requires analysis ID, load ID, dataset ID)
dattos datasets search 5192 3248 2476 --page-size 10 --folder-id 2473
# {"rows": {"data": [{"id": 1, "valor": 1500.00, "data": "2025-12-20", ...}]}, "totalRows": 1523}

# With pagination (default page-size is 50)
dattos datasets search 5192 3248 2476 --page-size 50 --page 2

# With filter (JSON string)
dattos datasets search 5192 3248 2476 --filter '{"column": "status", "value": "active"}'
```

### `dattos matching` — Reconciliation results

```bash
# Get matching execution info
dattos matching get 5192 3248 2475 --folder-id 2473
# {"id": 1, "status": "Completed", "matchedCount": 1523, ...}

# Get totals by status (matched, unmatched, pending)
dattos matching status 5192 3248 2475 --folder-id 2473
# {"Matched": 1523, "Unmatched": 47, "Pending": 0}

# Get matching summary report
dattos matching summary 5192 3248 2475 --folder-id 2473
# [{"column": "Account", "matched": 100, "unmatched": 5}, ...]
```

### `dattos reports` — Generate and download reports

```bash
# Generate a report (async — returns report ID to poll)
dattos reports generate 5192 --load-id 3248 --dataset-id 2476 --format csv
# {"id": 99, "status": "Processing"}

dattos reports generate 5192 --load-id 3248 --dataset-id 2476 --format xlsx
# Note: >1M rows only supports CSV format

# Check report generation status
dattos reports status 5192 --load-id 3248 --dataset-id 2476
# {"id": 99, "status": "Completed", "downloadUrl": "..."}

# Download a generated report
dattos reports download 5192 99 --output report.csv
# {"status": "downloaded", "file": "report.csv"}
```

### `dattos workflows` — Manage and execute workflows

```bash
# List workflows
dattos workflows list --page-size 10
dattos workflows list --name "Daily Recon"
# {"data": [{"Workflow": {"Id": 1, "Name": "Daily Recon"}, "ExecutionWorkflow": {...}}, ...], "totalItems": 5}

# Get workflow details
dattos workflows get 1
# {"Workflow": {"Id": 1, "Name": "Daily Recon", "IsActive": true}, "ExecutionWorkflow": {"Status": "Done", ...}}

# Start a workflow for a reference date
dattos workflows start 1 --date 2026-03-08
# {"status": "started", "workflowId": 1, "referenceDate": "2026-03-08"}

# Stop a running workflow execution
dattos workflows stop 10  # uses execution ID, not workflow ID
# {"status": "stopped", "executionId": 10}

# View execution history
dattos workflows history 1 --page-size 20
# {"data": [{"Id": 10, "Status": "Done", "ReferenceDate": "2026-03-07"}, ...], "totalItems": 15}

# View task logs
dattos workflows logs 55  # uses task ID from execution
# [{"message": "Starting import", "timestamp": "2026-03-07T10:00:00"}, ...]
```

### `dattos imports` — Monitor and manage imports

```bash
# List imports (with optional filters)
dattos imports list
dattos imports list --from 2026-01-01 --to 2026-03-31
dattos imports list --status Error
dattos imports list --include-canceled
# {"data": [{"Id": 1, "Status": "Success", "TotalSuccess": 100, ...}], "totalItems": 25}

# Get import details
dattos imports get 1
# {"Id": 1, "Status": "Success", "TotalSuccess": 100, "TotalRejections": 0, ...}

# View rejection details
dattos imports results 1
# {"data": [{"line": 5, "reason": "Invalid date format"}], "totalItems": 1}

# View import logs
dattos imports logs 1
# {"data": [{"message": "Processing started", "timestamp": "..."}], "totalItems": 3}

# Cancel an import
dattos imports cancel 1
# {"status": "canceled", "importId": 1}
```

### `dattos folders` — List, select and inspect folders

```bash
# List all folders as a tree (raw JSON)
dattos folders list

# Flat list — easier to find folder IDs
dattos folders list --flat
# [{"id": 2473, "code": "SIR", "name": "sirio", "fullPath": "Engenharia / sirio", "depth": 1}, ...]

# Interactive folder picker (TTY only)
dattos folders select

# Select folder by ID
dattos folders select 2473
# {"folder_id": "2473", "folder_name": "sirio", "status": "selected"}

# Show active folder
dattos folders current
# {"folder_id": "2473", "folder_name": "sirio"}
```

### `dattos users` — Manage users

```bash
# List all users (flat array)
dattos users list
dattos users list --search "john" --page-size 20
# [{"Id": 4, "UserCode": "david.barouh", "FullName": "David Barouh", "EmailAddress": "...", "Blocked": false, ...}]

# Get a user by ID
dattos users get 4
# {"Id": 4, "UserCode": "david.barouh", "FullName": "David Barouh", ...}

# List available Privacy Profiles (roles)
dattos users list-roles
# [{"Id": 1, "Name": "Administradores", ...}, {"Id": 2, "Name": "Engenharia", ...}]

# Get roles assigned to a user
dattos users roles 4
# [{"Id": 1, "Name": "Administradores", ...}, ...]

# Create a new user — Dattos sends an invitation email automatically
dattos users create --code "JDOE" --name "John Doe" --email "john.doe@company.com"
dattos users create --code "JDOE" --name "John Doe" --email "john.doe@company.com" --culture "en-US"
# {"Id": 80, "UserCode": "JDOE", "FullName": "John Doe", ...}

# Update a user (read-modify-write — only provided fields change)
dattos users update 80 --name "John P. Doe"
dattos users update 80 --culture "en-US"
dattos users update 80 --blocked   # block the user
dattos users update 80 --active    # unblock the user

# Assign Privacy Profiles — REPLACES all existing roles
dattos users set-roles 80 --role-id 1 --role-id 2
# {} (success)

# Delete a user — requires --yes (irreversible)
dattos users delete 80 --yes
# {"status": "deleted", "userId": 80}
```

### `dattos config` — Manage configuration

```bash
# Set config values (API key stored in OS keyring)
dattos config set api-url "https://your-instance.dattos.com.br/dattos.api"
dattos config set api-key "api-xxxxxxxx"
dattos config set folder-id "2473"

# Show current resolved configuration
dattos config show
# {"profile": "default", "api_url": "...", "api_key": "***xxxx", "folder_id": "2473", "folder_name": "sirio"}

# List profiles
dattos config profile list
# {"profiles": ["default", "staging"], "active": "default"}

# Switch profile
dattos config profile switch staging
```

### `dattos auth` — Authentication

```bash
# Log in interactively
dattos auth login

# Check auth status
dattos auth status
# {"method": "session", "profile": "default", "token": "abc12345***"}

# Log out
dattos auth logout
```

### `dattos doctor` — Diagnose configuration

```bash
dattos doctor
#   dattos-cli v0.7.0
#   profile: default
#
#     OK  Config  (https://your-instance.dattos.com.br/dattos.api)
#     OK  API reachable
#     OK  Auth
#     OK  Auth method  (API key)
#     OK  Folder  (sirio)
```

### `dattos version`

```bash
dattos version
# {"version": "1.0.0"}
```

## Usage Examples

### Example 1: First-time setup

```bash
# Configure credentials
dattos config set api-url "https://your-instance.dattos.com.br/dattos.api"
dattos config set api-key "api-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"

# Select a folder (interactive picker)
dattos folders select

# Or select by ID directly
dattos folders list --flat
dattos folders select 2473

# Verify everything works
dattos config show
dattos analyses list --page-size 5
```

### Example 2: Inspect reconciliation results

```bash
# Find the analysis by code
dattos analyses get-by-code "RECON-BANK"
# Note the analysisId (e.g., 5192)

# Check if data is loaded for a date
dattos analyses execution-status 5192 --date 2026-03-01
# {"status": "Loaded", "completedPercent": 100.0, ...}

# Get the load (returns load ID and dataset IDs)
dattos loads get 5192 2026-03-01
# Note loadId (3248) and datasetId (2476)

# Browse the data
dattos datasets search 5192 3248 2476 --page-size 10

# Check matching results
dattos matching status 5192 3248 2475
# {"Matched": 1523, "Unmatched": 47, "Pending": 0}
```

### Example 3: Export data for external analysis

```bash
# Generate a CSV report
dattos reports generate 5192 --load-id 3248 --dataset-id 2476 --format csv
# {"id": 99, "status": "Processing"}

# Wait for it to complete
dattos reports status 5192 --load-id 3248 --dataset-id 2476
# {"id": 99, "status": "Completed"}

# Download the file
dattos reports download 5192 99 --output reconciliation_2026-03-01.csv
# {"status": "downloaded", "file": "reconciliation_2026-03-01.csv"}
```

### Example 4: Find loaded dates for an analysis

```bash
# Which dates have data loaded?
dattos loads list-dates 4137 --from 2026-01-01 --to 2026-03-31 | jq -r '.[]'
# 2026-01-15T00:00:00
# 2026-02-15T00:00:00
# 2026-03-15T00:00:00
```

### Example 5: Composing with jq

```bash
# List analysis names and codes
dattos analyses list --page-size 100 | jq '.data[] | {code: .analysisCode, name: .analysisName}'

# Count unmatched items
dattos matching status 5192 3248 2475 | jq '.Unmatched'

# Get dataset column names
dattos loads get 5192 2026-03-01 | jq '.datasets[].columns[].name'

# Check if execution is done (useful in scripts)
STATUS=$(dattos analyses execution-status 5192 --date 2026-03-01 | jq -r '.status')
if [ "$STATUS" = "Loaded" ]; then echo "Ready"; fi
```

## Typical Agent Workflow

A typical AI agent workflow to inspect reconciliation results:

```bash
# 0. First time: select a folder
dattos folders select

# 1. List analyses
dattos analyses list --page-size 10

# 2. Check execution status
dattos analyses execution-status 5192 --date 2025-12-20

# 3. Get load details (returns load ID + dataset IDs)
dattos loads get 5192 2025-12-20

# 4. Search dataset rows
dattos datasets search 5192 3248 2476 --page-size 50

# 5. Check matching results
dattos matching status 5192 3248 2475

# 6. Export a report
dattos reports generate 5192 --load-id 3248 --dataset-id 2476 --format csv
```

## Output Format

All commands output **JSON to stdout**. Errors output structured JSON to **stderr**.

```bash
# Success — JSON to stdout
dattos analyses get 5192
# {"analysisId": 5192, "analysisCode": "PRD-7728", ...}

# Error — JSON to stderr, non-zero exit code
dattos analyses get 99999
# {"error": "not_found", "message": "A entidade procurada não existe.", "status": 404}
```

### Exit Codes

| Code | Meaning |
|------|---------|
| `0` | Success |
| `1` | API error (4xx/5xx) |
| `2` | Auth/config error (missing API key, invalid credentials) |

### Piping and Filtering

Because output is JSON, it composes naturally with `jq`:

```bash
# Get just the analysis names
dattos analyses list --folder-id 2473 | jq '.data[].analysisName'

# Get loaded dates as plain text
dattos loads list-dates 4137 --from 2025-01-01 --to 2026-12-31 --folder-id 2473 | jq -r '.[]'

# Check if execution is complete
dattos analyses execution-status 5192 --date 2025-12-20 | jq -r '.status'
```

## Error Handling

The CLI handles common API errors with descriptive messages:

| HTTP Status | Error Type | Description |
|-------------|------------|-------------|
| `401` | `auth_error` | Invalid or expired API key |
| `403` | `forbidden` | Insufficient permissions |
| `404` | `not_found` | Resource does not exist |
| `429` | `rate_limited` | Too many requests (auto-retried up to 3 times) |
| Other | `api_error` | General API error |

Rate limiting (HTTP 429) is handled automatically with exponential backoff (5s, 10s, 20s, max 30s).

## Using the SDK

The `dattos_sdk` package is included with the CLI and can be used directly in Python scripts, notebooks, or automations — no terminal needed.

### Quick Start

```python
from dattos_sdk import DattosClient, ApiError

client = DattosClient(
    base_url="https://your-instance.dattos.com.br/dattos.api",
    api_key="api-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
    folder_id="123",
)

# List analyses
analyses = client.get("/api/Analysis", params={"pageInfo.page": 0, "pageInfo.pageSize": 10})
for item in analyses["data"]:
    print(item["analysisName"])

# Handle errors
try:
    client.get("/api/Analysis/99999")
except ApiError as e:
    print(f"Error {e.status_code}: {e.detail}")
```

### Using Existing CLI Config

If you've already configured the CLI (`dattos config set ...`), the SDK can load those settings automatically:

```python
from dattos_sdk import DattosClient, get_config

cfg = get_config()  # loads from active profile, env vars, keyring
client = DattosClient(base_url=cfg.api_url, api_key=cfg.api_key, folder_id=cfg.folder_id)
```

### Caching

Enable local caching on GET requests to avoid redundant API calls:

```python
from pathlib import Path
from dattos_sdk import DattosClient

client = DattosClient(
    base_url="https://...", api_key="...",
    cache_dir=Path("~/.dattos/cache/default").expanduser(),
)

# First call hits the API, second returns from cache (5-minute TTL)
data = client.get("/api/Analysis", cache=True, cache_ttl=300)
data = client.get("/api/Analysis", cache=True)  # instant, from cache
```

### SDK API Reference

**`DattosClient(base_url, api_key, folder_id="", cache_dir=None)`**

| Method | Description |
|--------|-------------|
| `get(path, cache=False, cache_ttl=300, **kwargs)` | GET request. Returns `dict \| list`. |
| `post(path, **kwargs)` | POST request. Invalidates related cache. |
| `put(path, **kwargs)` | PUT request. Invalidates related cache. |
| `delete(path, **kwargs)` | DELETE request. Invalidates related cache. |
| `upload(path, data: bytes, **kwargs)` | Upload binary data. |
| `download(path, **kwargs)` | Download file. Returns `httpx.Response`. |

**`DattosConfig`** (returned by `get_config()`): `api_url`, `api_key`, `folder_id`, `folder_name`

**`ApiError`**: `status_code` (int), `detail` (str), `url` (str)

**`FileCache(cache_dir)`**: `get(path, params)`, `set(path, params, data, ttl)`, `invalidate(pattern)`, `clear()`

## Versioning

This project follows [Semantic Versioning](https://semver.org/):

- **Patch** (1.0.x): bug fixes, no API changes
- **Minor** (1.x.0): new features, backwards compatible
- **Major** (x.0.0): breaking changes to the SDK public API

The public API surface (frozen at 1.0) includes: `DattosClient`, `DattosConfig`, `ApiError`, `get_config`, `FileCache`.

## Project Structure

```
dattos-cli/
├── src/dattos_cli/          # CLI layer (thin wrappers over SDK)
│   ├── __init__.py          # Version
│   ├── main.py              # Typer app, client factory, error handling
│   ├── client.py            # Re-export shim → dattos_sdk.client
│   ├── config.py            # Re-export shim → dattos_sdk.config
│   ├── credentials.py       # Re-export shim → dattos_sdk.credentials
│   ├── output.py            # JSON output helpers (stdout/stderr)
│   └── commands/            # 19 command modules
├── src/dattos_sdk/          # SDK layer (reusable library)
│   ├── __init__.py          # Public API exports
│   ├── client.py            # HTTP client with retry, cache integration
│   ├── config.py            # Config loading (profiles, env vars, keyring)
│   ├── credentials.py       # OS keyring abstraction
│   ├── exceptions.py        # ApiError
│   ├── cache.py             # File-based TTL cache
│   └── py.typed             # PEP 561 type checker support
├── tests/                   # 216 unit + 13 contract + 9 E2E tests
├── docs/                    # Design specs and implementation plans
├── pyproject.toml
└── .gitignore
```

## Development

### Branch Workflow

All development happens on `develop`. The `main` branch is protected — changes require a PR.

```bash
git checkout develop
# make changes...
git push origin develop
# create PR: develop → main
```

### Setup

```bash
pip install -e .
pip install pytest

# Run tests
pytest -v

# Run a specific test
pytest tests/test_analyses.py -v
```

### Tech Stack

- **Python 3.11+**
- **[Typer](https://typer.tiangolo.com/)** — CLI framework with type hints and auto-generated help
- **[httpx](https://www.python-httpx.org/)** — HTTP client with MockTransport for testing
- **[keyring](https://pypi.org/project/keyring/)** — OS credential store (optional)
- **[hatchling](https://hatch.pypa.io/)** — Build backend

## Roadmap

- ~~**v0.2** — Execution commands~~ ✅
- ~~**v0.3** — User management~~ ✅
- ~~**v0.4** — Interactive folder selection~~ ✅
- ~~**v0.5** — Developer experience~~ ✅
- ~~**v0.6** — TTY tables, --raw flag~~ ✅
- ~~**v0.7** — Security & multi-environment (keyring, login, profiles)~~ ✅
- ~~**v0.8** — Full API coverage (connectors, config packs, ETL)~~ ✅
- ~~**v0.9** — Robustness (SDK extraction, cache, contract tests, bulk ops)~~ ✅
- ~~**v1.0** — Stable release (semver, E2E tests, SDK docs)~~ ✅

See [ROADMAP.md](docs/ROADMAP.md) for details.
