---
name: loony-test-validate
description: Test and validate a loony pipeline — fix type mismatches, schema drift, transform errors
allowed-tools: Bash(loony *), Read, Write, Edit
---

# Test & Validate a Loony Pipeline

## Quick Test

```bash
# Test a single script
loony test <script_name>

# Test everything (all scripts + transforms + views)
loony test --all
```

## Reading Validation Output

### Pass (✓)
```
✓ Table raw_items exists
✓ Primary key (id) is unique
```
No action needed.

### Warning (⚠)
```
⚠ Column 'value': declared integer, actual varchar — castable with NULLIF for values: ["-"]
  Suggested: COALESCE(NULLIF(value::text, '-')::integer, 0)
```
The column type doesn't match schema.yaml but is castable. Use the suggested SQL in your transforms.

### Error (✗)
```
✗ Missing columns: ['loaded_at']
✗ Primary key (id, date) has 15 duplicates
```
Must fix before deploying.

## Common Fixes

### Type Mismatch (varchar vs integer)
**Cause**: API returns numbers as strings or uses "-" for missing values. dlt infers varchar.

**Fix in transforms**:
```sql
-- In 001_staged.sql
COALESCE(NULLIF(column_name::text, '-')::integer, 0)
```

### Missing Column
**Cause**: schema.yaml declares a column the script doesn't produce.

**Fix**: Either:
- Add the column to the script's dlt resource
- Remove it from schema.yaml
- Add it in the staged transform: `now() AS loaded_at`

### Primary Key Duplicates
**Cause**: Primary key in schema.yaml doesn't uniquely identify rows.

**Fix**: Either:
- Add more columns to the primary key
- Add a DISTINCT ON in the staged transform
- Use `write_disposition="merge"` so dlt deduplicates

### Transform SQL Error
**Cause**: Usually a type mismatch between raw and staged columns, or referencing a table that doesn't exist yet.

**Fix**:
- Check the error message — it tells you which statement and line
- Ensure raw tables exist before staged transforms reference them
- Cast types explicitly — never rely on implicit casts from API data

## Validation Checklist Before Deploy

1. `loony test --all` passes
2. All raw tables have rows
3. All staged tables have rows
4. All materialized views have rows
5. No primary key duplicates
6. Type mismatches are handled in transforms (warnings OK, errors not)
7. schema.yaml measures reference real columns in the views
