# Type Promotion + Chumsky Parser Redesign

Date: 2026-02-24

## Goals

- Replace the hand-written parser with a chumsky 0.12 Pratt parser that matches docs/grammar.md and uses Rich errors + recovery.
- Enforce default literal typing rules from docs/type-system.md (min signed int that fits; minimal exact IEEE float/complex).
- Make type promotion concrete: insert explicit casts on both inputs before every binary op/compare and before if-branch merge.
- Align semantics across interpreter, const-folding, and codegen, including integer-only floor division for // and integer %.
- Add spec-driven edge tests (parser, lowering, runtime semantics) as behavior baseline.

## Scope

In scope:

- Parser replacement (src/parse/mod.rs) using chumsky Pratt + Rich errors + recovery.
- Lowering changes in src/ir/lower.rs to insert casts before ops and if merges.
- Runtime semantics: interpreter, const-fold, codegen cast support, and integer // / % semantics.
- Documentation alignment: docs/grammar.md default literal type rules updated to match docs/type-system.md.
- Tests: additional edge cases for parser, lowering, and integer division/remainder semantics.

Out of scope:

- Any new language features beyond docs/grammar.md.
- Changes to public AST shape (keep current ast.rs / error.rs interfaces stable).

## Current Issues and Constraints

- Parser is currently hand-written; docs require chumsky with Pratt/precedence and advanced error handling.
- docs/type-system.md and docs/grammar.md disagree on default literal types; tests must be the behavior baseline.
- Lowering computes promotion but does not insert casts; interpreter and const-fold rely on f64 paths that are incorrect for integers.
- codegen treats Node::Cast as no-op, which breaks explicit cast semantics.

## Proposed Design

### 1) Parser (chumsky Pratt + Rich errors)

Tokenizer:

- Use chumsky tokenization that preserves span ranges and produces a token stream.
- Distinguish keywords from identifiers (if/then/else/cast/true/false/e/pi).

Pratt parser:

- Match operator table in docs/grammar.md.
- Right-associative **, prefix -/!, non-assoc comparisons, left-assoc && and ||.
- Atom forms: cast<dtype>(expr), call, tuple (>=2 items), grouping, literal, const_sym, ident.

Error handling:

- Use Rich errors with labels for missing tokens and unexpected tokens.
- Add recover_with sync points (newline, ')', ',') to improve recovery on syntax errors.

### 2) Lowering (cast-before-op semantics)

Binary ops and comparisons:

- Before each binary op: compute out_ty = promote(lhs_ty, rhs_ty) after kind checks for / and //.
- Insert Node::Cast(out_ty, lhs) and Node::Cast(out_ty, rhs) when needed.
- Build the operation node with the casted operands. Comparison nodes still return bool.

If-expressions:

- out_ty = promote(then_ty, else_ty); insert casts on both branches before constructing IfElse.
- IfElse node dtype is out_ty.

Diagnostics:

- Promotion info emitted for each implicit promotion.
- Warning for u64 + SInt promotion per docs/type-system.md.

### 3) Runtime/Optimization/Codegen semantics

Interpreter:

- Implement Node::Cast as a real cast on values.
- Comparisons operate on aligned numeric values (no enum structural equality).
- Implement integer-only // (floor division) and % with the invariant:
  - a == (a//b) * b + (a % b)
  - remainder sign matches divisor.

Const-fold:

- Mirror interpreter semantics for // and %.
- Avoid f64 for integer-only ops to prevent precision loss.

Codegen:

- Emit explicit casts for Node::Cast (not a no-op) in Python and Rust backends.

### 4) Documentation alignment

- Update docs/grammar.md to match docs/type-system.md default literal typing rules.

### 5) Tests

Parser:

- Comparison chaining error points at the second comparator.
- Recovery on missing then/else/)/comma.
- Tuple must have >= 2 items.
- 2j rejected (requires 2.0j or 2e0j).

Lowering:

- Cast-before-op for comparisons and if-merge.
- Promotions into complex.
- Equality between int/float behaves by numeric cast.

Runtime/const-fold/codegen:

- // and % on negatives and mixed signed widths.
- Integer-only enforcement for //.

## Risks and Mitigations

- Parser swap may affect existing error spans: mitigate with Rich errors and span tests.
- Integer-only semantics for // and % could reveal previously masked behavior: add explicit tests to lock in spec behavior.
- Cast-before-op could surface latent backend assumptions: fix Node::Cast in interp and codegen in the same change.

## Success Criteria

- Parser uses chumsky Pratt and passes existing tests plus new edge tests.
- Default literal typing matches docs/type-system.md.
- Cast-before-op is visible in IR and respected by interpreter/const-fold/codegen.
- Integer // and % obey floor-division semantics and are rejected for non-integer kinds.
- All tests and clippy pass.
