{#- Context variables:
   - company: str (e.g., "ACME Bank")
   - industry: str (e.g., "fsi")
   - catalog: str (e.g., "acme_fsi_demo")
   - bundle_name: str (e.g., "acme-bank-fsi-demo")
   - app_name: str (e.g., "acme-bank-fsi-app")
   - warehouse_id: str
   - workspace_host: str
   - databricks_profile: str | None
   - lakebase_url: str | None
   - notification_email: str
-#}
# {{ demo_title }}

An end-to-end Databricks demo showcasing the full lakehouse platform for **{{ display_name }}** in the **{{ industry | upper }}** industry vertical.

Built with [exokit](https://github.com/your-org/exokit) and designed to be completed incrementally with [Claude Code](https://claude.ai/claude-code).

## What This Demo Includes

| Layer | Component | Status |
|-------|-----------|--------|
| 1 | **Data Generation** — Structured tables (Spark + Faker) + unstructured documents (PDFs for RAG) | Pending |
| 2 | **Medallion Pipelines** — Bronze (Auto Loader) → Silver (quality constraints) → Gold (business metrics) via DLT/SDP | Pending |
| 3 | **ML Models** — Feature engineering + AutoML training + model registration in Unity Catalog | Pending |
| 4 | **Dashboards** — AI/BI Lakeview dashboards + Genie Spaces for natural language analytics | Pending |
| 5 | **Showcase App** — Full-stack React + FastAPI application on Databricks Apps | Pending |
| 6 | **AI Agents** — Domain-specific agents using Databricks Agent Framework | Pending |

## Prerequisites

| Tool | Required | Check |
|------|----------|-------|
| Databricks CLI | >= 0.200 | `databricks --version` |
| Python | >= 3.11 | `python3 --version` |
| uv | >= 0.4 | `uv --version` |
| Node.js | >= 18 | `node --version` |
| Claude Code | latest | `claude --version` |

## Databricks Configuration

| Setting | Value |
|---------|-------|
| **Workspace** | `{{ workspace_host }}` |
| **Unity Catalog** | `{{ catalog }}` |
| **SQL Warehouse** | `{{ warehouse_id }}` |
{% if databricks_profile %}| **CLI Profile** | `{{ databricks_profile }}` |
{% endif %}{% if lakebase_url %}| **Lakebase** | Configured (see `.env`) |
{% endif %}| **Bundle Name** | `{{ bundle_name }}` |

## Quick Start

### 1. Set up environment

```bash
# Copy environment file and fill in your Lakebase URL if needed
cp .env.example .env
```

### 2. Plan with Claude Code

```bash
# Open Claude Code in this directory
claude

# First: create the implementation plan
/plan
```

`/plan` will:
1. Research {{ display_name }} and {{ industry | upper }} industry patterns
2. Ask you clarifying questions about the demo scope
3. Design schemas, entities, and ML targets
4. Create a detailed implementation plan for your approval
5. Apply the plan to scaffold placeholders (job YAML, pipeline config, variables)

### 3. Build wave by wave

```bash
# After the plan is approved, build each wave
/build-next
```

`/build-next` reads the manifest, finds the next pending wave, and builds it. Run it repeatedly to advance through all 5 waves.

### 4. Deploy

```bash
# Full deployment (lakehouse + app + Genie)
./scripts/deploy-all.sh dev

# Or separately:
./scripts/deploy-lakehouse.sh dev    # Data gen → pipeline → ML → Genie
./scripts/deploy-app.sh dev          # Build → deploy → SP permissions
```

## Redeployment

`exokit init` deploys the hello-world demo automatically. To redeploy after changes:

### Full redeploy

```bash
./scripts/deploy-all.sh dev
```

### Step-by-step redeploy

```bash
# 1. Deploy bundle (upload code + create resources)
databricks bundle deploy -t dev --profile {{ databricks_profile or 'DEFAULT' }}

# 2. Run data generation
databricks bundle run -t dev {{ bundle_name | replace('-', '_') }}_data_generation \
  --profile {{ databricks_profile or 'DEFAULT' }}

# 3. Run DLT pipeline (bronze → silver → gold)
databricks bundle run -t dev {{ bundle_name | replace('-', '_') }}_dlt_pipeline \
  --profile {{ databricks_profile or 'DEFAULT' }}

# 4. Run ML training
databricks bundle run -t dev {{ bundle_name | replace('-', '_') }}_ml_training \
  --profile {{ databricks_profile or 'DEFAULT' }}

# 5. Create Genie Space
python3 scripts/setup-genie.py --profile {{ databricks_profile or 'DEFAULT' }}

# 6. Deploy app (build → upload → deploy → grant permissions)
./scripts/deploy-app.sh dev
```

### Partial redeploy (skip to a specific step)

```bash
# Skip data gen, start from pipeline
./scripts/deploy-lakehouse.sh dev --skip-to pipeline

# Skip pipeline, start from ML
./scripts/deploy-lakehouse.sh dev --skip-to ml

# Skip ML, just create Genie Space
./scripts/deploy-lakehouse.sh dev --skip-to genie

# App only (skip lakehouse entirely)
SKIP_APP=false ./scripts/deploy-app.sh dev
```

## Project Structure

```
{{ bundle_name }}/
├── CLAUDE.md                    # Claude's operating manual
├── README.md                    # This file
├── databricks.yml               # Asset Bundle configuration
├── conf/
│   ├── demo_manifest.json       # Build plan — waves, schemas, decisions
│   └── catalog_schema.json      # Unity Catalog structure
├── .claude/
│   ├── agents/                  # Specialist agents (lakehouse, app, dashboard, reviewer)
│   ├── commands/                # /plan, /build-next, /deploy, /talk-track
│   └── skills/                  # Databricks patterns and best practices
├── src/
│   ├── data_generation/
│   │   ├── structured/          # Spark + Faker table generators
│   │   └── unstructured/        # PDF/document generators for RAG
│   ├── pipelines/
│   │   ├── bronze/              # STREAMING TABLE + read_files
│   │   ├── silver/              # STREAMING TABLE + quality constraints
│   │   └── gold/                # LIVE TABLE + business aggregations
│   ├── ml/                      # Feature engineering + model training
│   ├── dashboards/              # Lakeview definitions + deploy utilities
│   └── agents/                  # Databricks Agent Framework
├── apps/
│   └── {{ app_name }}/          # Full-stack showcase app
│       ├── backend/             # FastAPI (3-layer: api/services/repositories)
│       └── frontend/            # React + TypeScript + shadcn/ui
├── resources/
│   ├── jobs/                    # Databricks job definitions
│   └── pipelines/               # DLT pipeline definitions
├── docs/
│   ├── IMPLEMENTATION_PLAN.md   # Detailed build plan (created by /build-next)
│   ├── PROGRESS.md              # Wave completion tracker
│   ├── TALK_TRACK.md            # Demo presentation narrative
│   └── supplements/             # Architecture, coding standards, wave playbook
└── scripts/
    ├── deploy-all.sh            # Full deployment (lakehouse + app + Genie)
    ├── deploy-lakehouse.sh      # Data gen → pipeline → ML → Genie
    ├── deploy-app.sh            # Build → deploy → SP permissions
    ├── setup-genie.py           # Create Genie Space via REST API
    ├── validate.sh              # Pre-deployment checks
    ├── get_job_errors.py        # Debug failed job runs
    └── ensure-mcp.sh            # Auto-install MCP server
```

## Building Wave by Wave

Progress is tracked in `docs/PROGRESS.md` and `conf/demo_manifest.json`.

| Wave | What Gets Built | Claude Agent |
|------|-----------------|-------------|
| **Wave 1** | Data generation (structured + unstructured), UC setup | lakehouse-dev |
| **Wave 2** | DLT/SDP pipelines: bronze → silver → gold | lakehouse-dev |
| **Wave 3** | Feature engineering, ML models, dashboards, Genie | lakehouse-dev + dashboard-dev |
| **Wave 4** | Full-stack showcase app (FastAPI + React) | app-dev |
| **Wave 5** | AI agents, talk track, final polish | lakehouse-dev |

Each wave is built incrementally with quality gates. Use `/plan` first, then `/build-next` to advance.

## Key Commands

| Command | What It Does |
|---------|-------------|
| `/plan` | Research, design schemas, create implementation plan (run first) |
| `/build-next` | Build the next pending wave (run after `/plan` is approved) |
| `/deploy` | Deploy current state to Databricks |
| `/talk-track` | Generate demo presentation narrative |

## App Architecture

The showcase app (`apps/{{ app_name }}/`) uses production-grade patterns:

```
HTTP Request → Router (thin) → Service (business logic) → Repository (data access) → Lakebase
```

| Layer | Technology |
|-------|-----------|
| Backend | FastAPI + SQLAlchemy 2.0 + Lakebase (Postgres) |
| Frontend | React 19 + TypeScript + Vite + Tailwind + shadcn/ui |
| Server State | TanStack Query v5 |
| Client State | Zustand v5 |
| API Client | Auto-generated from OpenAPI spec |

## Useful Links

- **Workspace**: [{{ workspace_host }}]({{ workspace_host }})
{% if databricks_profile %}- **CLI Profile**: `{{ databricks_profile }}` (in `~/.databrickscfg`)
{% endif %}- **Unity Catalog**: `{{ catalog }}` → schemas: raw, analytics, ml
- **Bundle docs**: [Databricks Asset Bundles](https://docs.databricks.com/en/dev-tools/bundles/index.html)
- **DLT docs**: [Delta Live Tables](https://docs.databricks.com/en/delta-live-tables/index.html)

---

*Generated by [exokit](https://github.com/your-org/exokit) on {{ "now" | default("today") }}.*
