# Sprint 1 — TSCN Parser: Implementation Plan

## 1. What We're Building

A complete parser for Godot 4.x `.tscn` (text scene) files — the most critical component of GodotIQ. The parser reads a `.tscn` file from disk, extracts all structural information (header, resources, nodes, connections), decomposes Transform3D matrices into position/rotation/scale, reconstructs the full parent-child hierarchy, and returns a rich typed data structure that 10+ downstream MCP tools will consume.

### Why This Matters

Every spatial, code, and animation tool in GodotIQ depends on understanding scene structure. Without the TSCN parser, none of the planned 30 tools can function. This is Sprint 1 because it's the foundation everything else builds on.

## 2. Project Context

GodotIQ is a Python MCP server (`src/godotiq/` package, imports as `from godotiq.X import Y`). Sprint 0 is complete: the package installs, the MCP server runs with a ping tool, and 10 tests pass. Existing placeholder code lives in `src/godotiq/parsers/tscn_parser.py` with basic `TscnNode`/`TscnScene` dataclasses and a `parse_tscn()` stub that raises `NotImplementedError`.

**Constraints:**
- Python 3.10+, zero external dependencies (stdlib only)
- Must not break Sprint 0 tests (update test expectations for API changes)
- Performance target: parse a 4711-line scene file in < 200ms
- All malformed input handled gracefully (skip + warn via `logging` module, never crash)
- Files opened with `encoding="utf-8"` explicitly
- 2D transform decomposition (Transform2D) is explicitly deferred to a future sprint

## 3. Data Model

### 3.1 New Dataclasses to Create

**`SceneHeader`** — Parsed `[gd_scene]` line:
- `format: int` — 3 or higher
- `load_steps: int | None`
- `uid: str | None`

**`ExtResource`** — One `[ext_resource]` entry:
- `id: str` — reference ID used by `ExtResource("id")`
- `type: str` — resource class ("Script", "PackedScene", "Texture2D", etc.)
- `path: str` — `res://` path to the resource file
- `uid: str | None`

**`SubResource`** — One `[sub_resource]` block:
- `id: str`
- `type: str`
- `properties: dict[str, object]`

**`Connection`** — One `[connection]` entry:
- `signal: str`
- `from_node: str` — node path of emitter
- `to_node: str` — node path of receiver
- `method: str`
- `flags: int` — default 0
- `binds: list[object]` — default empty

### 3.2 Extended `TscnNode`

Add these fields to the existing `TscnNode` dataclass (preserve `name`, `type`, `properties`, `children`):

- `parent: str` — parent path string ("." for root children, "" for root itself)
- `instance: str | None` — ExtResource ID if this is an instanced scene
- `unique_id: int | None`
- `groups: list[str]`
- `script: str | None` — ExtResource ID if node has a script
- `position: tuple[float, float, float] | None` — from Transform3D decomposition
- `rotation: tuple[float, float, float] | None` — YXZ Euler angles in radians
- `scale: tuple[float, float, float] | None`
- `metadata: dict[str, object]` — properties with `metadata/` prefix, stored without the prefix

The `properties` dict stores all remaining node properties that don't map to a dedicated field. Properties with `metadata/` prefix are moved to `metadata` instead.

### 3.3 Extended `TscnScene`

Replace the minimal placeholder with (removing the old `resources` field):

- `header: SceneHeader`
- `ext_resources: dict[str, ExtResource]` — keyed by ID
- `sub_resources: dict[str, SubResource]` — keyed by ID
- `root: TscnNode | None` — the tree root (node without parent)
- `nodes: list[TscnNode]` — flat list of all nodes in file order
- `connections: list[Connection]`

Plus query methods (Section 6).

### 3.4 API Change

The `parse_tscn` signature changes from `parse_tscn(content: str) -> TscnScene` to `parse_tscn(path: str) -> TscnScene`. The function reads the file at `path` (with `encoding="utf-8"`) and returns the parsed scene. Sprint 0 tests that import `parse_tscn` and expect `NotImplementedError` must be updated.

## 4. Parser Architecture

### 4.1 Overview

Two-pass line-by-line state machine:

**Pass 1 (linear scan):** Read all lines, classify into sections, parse headers and properties. Output: flat lists of parsed sections (resources, nodes, connections). Handles multiline values via bracket/brace depth tracking.

**Pass 2 (tree construction):** Reconstruct parent-child hierarchy from the flat node list using the `parent` path attribute.

### 4.2 State Machine

States correspond to section types:

```
INITIAL → FILE_HEADER → EXT_RESOURCE → SUB_RESOURCE → NODE → CONNECTION → EDITABLE
```

Transitions happen when a new `[section_type ...]` header is encountered. Any section header triggers a transition regardless of current state (the state simply tracks which section is being populated). Lines between headers are property lines belonging to the current section. Blank lines and comment lines (starting with `;`) are skipped.

### 4.3 Section Header Parsing

Each `[section_type key=value key=value ...]` line has:
- A section type identifier (the first word after `[`)
- Zero or more `key=value` attributes

Header attribute values can be:
- Quoted strings: `name="NodeName"`
- Bare integers: `format=3`, `unique_id=12345`
- Array literals: `groups=["enemy", "damageable"]` — parse as list of strings
- ExtResource references: `instance=ExtResource("2_model")` — parse via value parser to extract the ID string

Use a pre-compiled regex to extract the section type and then iterate over `key=value` pairs. For array and ExtResource values in headers, invoke the value parser on the attribute value string.

### 4.4 Property Line Parsing

Lines like `property_name = value` below a section header. **Delimiter is ` = ` (space-equals-space)**, not bare `=`. Split on the first ` = ` occurrence. The value side is parsed by the value parser (Section 5).

Properties with `metadata/` prefix on nodes are extracted into the node's `metadata` dict with the prefix stripped.

### 4.5 Multiline Value Accumulation

Some property values span multiple lines, particularly `_surfaces = [{...}]` blocks in sub_resources. The parser must handle this:

1. When a property line's value has unbalanced brackets/braces/parentheses, continue accumulating subsequent lines until balanced.
2. Track depth for: `(` `)`, `[` `]`, `{` `}` — incrementing depth on open, decrementing on close.
3. A line starting with `[` followed by a section keyword (gd_scene, ext_resource, sub_resource, node, connection, editable) is always a new section header, never a continuation — check for this before depth tracking.
4. Once brackets are balanced, join accumulated lines and parse the complete value.

This is critical for `main.tscn` which contains 52+ multiline sub_resource property blocks.

### 4.6 Node-Specific Handling

When parsing a `[node]` section:
1. Extract `name`, `type`, `parent`, `instance`, `groups`, `unique_id` from the header (using header attribute parser, including array and ExtResource value types)
2. Parse property lines:
   - `script = ExtResource("id")` → extract ID, store in `node.script`
   - `transform = Transform3D(...)` → decompose into position/rotation/scale
   - `metadata/key = value` → store in `node.metadata["key"]`
   - Everything else → store in `node.properties`

### 4.7 Tree Reconstruction (Pass 2)

After all nodes are parsed into a flat list:
1. The node without a `parent` attribute (empty string) is the root
2. Build a name-based path → node lookup. The root's path is its name. For non-root nodes: if parent is ".", the node's full path is `root_name/node_name`; if parent is "A/B", the node's full path is `root_name/A/B/node_name`.
3. Assign each non-root node to its parent's `children` list
4. Set `scene.root` to the root node

**Root node lookup**: `find_node("")` or `find_node(root_name)` returns the root. `find_node("Child")` finds a direct child of root.

## 5. Value Parser

A dedicated module (`value_parser.py`) handles parsing the right-hand side of property assignments.

### 5.1 Type Detection Strategy

Pre-compiled regex patterns detect value types. **Scalar keywords (`true`, `false`, `null`) must be exact full-string matches, not prefix matches.**

| Pattern | Type | Parsing |
|---------|------|---------|
| `Vector2(...)` | Vector2 | Extract 2 floats → tuple |
| `Vector2i(...)` | Vector2i | Extract 2 ints → tuple |
| `Vector3(...)` | Vector3 | Extract 3 floats → tuple |
| `Vector3i(...)` | Vector3i | Extract 3 ints → tuple |
| `Vector4(...)` | Vector4 | Extract 4 floats → tuple |
| `Color(...)` | Color | Extract 4 floats → tuple |
| `Rect2(...)` | Rect2 | Extract 4 floats → tuple |
| `AABB(...)` | AABB | Extract 6 floats → tuple |
| `Transform3D(...)` | Transform3D | Extract 12 floats → decompose (see 5.2) |
| `Transform2D(...)` | Transform2D | Extract 6 floats → tuple (no decomposition, deferred) |
| `Basis(...)` | Basis | Extract 9 floats → tuple |
| `Plane(...)` | Plane | Extract 4 floats → tuple |
| `Quaternion(...)` | Quaternion | Extract 4 floats → tuple |
| `ExtResource("id")` | ExtResource ref | Extract string ID |
| `SubResource("id")` | SubResource ref | Extract string ID |
| `NodePath("path")` | NodePath | Extract string |
| `StringName(&"name")` | StringName | Extract string |
| `Packed*Array(...)` | Packed array | Store entire string as-is (no decode) |
| `{...}` | Dictionary | Parse key-value pairs recursively |
| `[...]` | Array | Parse elements recursively |
| `"..."` | String | Strip quotes, handle escapes |
| `true` | bool | Python `True` (exact match only) |
| `false` | bool | Python `False` (exact match only) |
| `null` | NoneType | Python `None` (exact match only) |
| Numeric | int or float | Parse as int if no decimal/exponent, else float |

The dispatch order: check for known type prefixes first (fast string `startswith`), then check scalars as exact matches, then attempt numeric parsing.

### 5.2 Dictionary Value Parsing

Godot dictionaries appear in sub_resource properties (e.g., `_surfaces`):
```
{
"key": value,
"key2": value2
}
```

Parse by: stripping outer `{` `}`, splitting on comma-separated key-value pairs (respecting nesting depth), parsing each key (always a quoted string) and each value (recursive call to `parse_value`).

For the `_surfaces` pattern specifically: it's an array `[{...}]` containing dictionaries. The array parser calls the dictionary parser, which recursively calls `parse_value` for each dictionary value — handling nested types like `AABB(...)`, `PackedByteArray(...)`, `SubResource(...)`, etc.

### 5.3 Transform3D Decomposition

This is the most critical piece. Given 12 floats from `Transform3D(...)`:

```
Transform3D(b00, b10, b20, b01, b11, b21, b02, b12, b22, ox, oy, oz)
```

**Step 1 — Extract position:** `(ox, oy, oz)` — the last 3 values.

**Step 2 — Reconstruct basis matrix (row-major):**
The values are column-major, so the row-major matrix is:
```
row 0: [b00, b01, b02]
row 1: [b10, b11, b12]
row 2: [b20, b21, b22]
```

**Step 3 — Clean floating-point noise:** Any value with `abs(v) < 1e-7` snaps to `0.0`. This handles Godot's float32 precision artifacts like `-3.059797e-08`.

**Step 4 — Extract scale with determinant sign check:**
Column vector lengths give absolute scale:
```
sx = sqrt(b00² + b10² + b20²)
sy = sqrt(b01² + b11² + b21²)
sz = sqrt(b02² + b12² + b22²)
```

Then compute the determinant of the 3x3 basis matrix. If the determinant is negative, negate `sx` (conventional choice). This correctly handles mirrored/reflected objects.

**Step 5 — Normalize basis:** Divide each column by its scale (using signed scale from Step 4) to get a pure rotation matrix. Guard against zero scale (skip normalization for that axis, treat as identity for that column).

**Step 6 — YXZ Euler decomposition:** Apply Godot's exact algorithm from `basis.cpp`:
- Examine `rows[1][2]` of the normalized matrix
- Normal case: `x = asin(-m12)`, `y = atan2(m02, m22)`, `z = atan2(m10, m11)`
- Gimbal lock (|m12| ≈ 1): set `z = 0`, compute `y` from alternative elements
- Epsilon for gimbal lock detection: `0.00000025` (matches Godot PR #102144)

**Step 7 — Clean output angles:** Snap near-zero angles (< 1e-10 rad) to 0.0.

Return `(position, rotation, scale)` as three tuples.

## 6. Query Methods on TscnScene

These methods support downstream tools and are needed for integration tests. **All methods handle missing resource IDs gracefully** (return empty results rather than raising KeyError).

**`find_node(path: str) -> TscnNode | None`**
Given a node path like "Building/LockedZones/PostProcessingZone", traverse the tree from root and return the matching node. Split path by "/" and walk children by name. Empty string or root node name returns the root. Paths with special characters (like `@CSGBox3D@106885`) work — no special handling needed, just match by name.

**`get_nodes_by_group(group: str) -> list[TscnNode]`**
Filter `self.nodes` for nodes where `group in node.groups`.

**`get_nodes_by_type(type_name: str) -> list[TscnNode]`**
Filter `self.nodes` for nodes where `node.type == type_name`.

**`get_all_scripts() -> list[str]`**
Collect `ext_resources[node.script].path` for nodes with a script, skipping any nodes whose script ID is not found in ext_resources. Return unique resource paths.

**`get_all_instances() -> list[str]`**
Collect `ext_resources[node.instance].path` for instanced nodes, skipping any whose instance ID is not found. Return unique resource paths.

## 7. File Organization

```
src/godotiq/parsers/
├── __init__.py          # (existing)
├── tscn_parser.py       # Dataclasses, parse_tscn(), TscnScene with query methods, state machine
└── value_parser.py      # NEW: parse_value(), decompose_transform3d(), type-specific parsers
```

`tscn_parser.py` contains the state machine, dataclass definitions, and `parse_tscn()` entry point.
`value_parser.py` contains all value parsing logic including Transform3D decomposition. Keeps tscn_parser.py focused on structure.

## 8. Sprint 0 Test Updates

The `parse_tscn` signature change (content → path) requires updating:
- `tests/test_server.py: test_parser_stubs_raise` — `parse_tscn` no longer raises `NotImplementedError`. Update the test to verify `parse_tscn` is callable (or test that it raises `FileNotFoundError` for nonexistent paths, confirming the new path-based API).
- Integration test `test_scaffolding_int.py` imports `parse_tscn` — import still works, just the behavior changes.

## 9. Test Strategy

### 9.1 Test Fixtures (in `tests/fixtures/`)

Four `.tscn` files. `furniture_slot_placeholder.tscn` needs to be created per the user's spec (37 lines, 6 nodes, multiple sub_resources with Color, materials). The other three already exist:
- `worker.tscn` — 19 lines, 4 nodes (simple hierarchy, script, SubResource refs)
- `fdm_printer.tscn` — 29 lines, 6 nodes (instances, groups, uniform scale)
- `furniture_slot_placeholder.tscn` — 37 lines, 6 nodes (to be created: 3 sub_resources, Color, material refs)
- `main.tscn` — 4711 lines, 876 nodes (stress test)

### 9.2 Unit Tests (`tests/test_parsers/test_tscn_parser.py`) — 23 tests

Organized by feature:

**Header parsing (2):** format=3 from worker.tscn, format from main.tscn (verify actual format during implementation)

**Resource parsing (2):** ext_resources from fdm_printer.tscn (count + contents), sub_resources from furniture_slot.tscn (count + properties)

**Node parsing (3):** basic node count/names from worker.tscn, hierarchy verification from fdm_printer.tscn, deep nesting from main.tscn

**Groups (2):** single group from fdm_printer.tscn, multiple groups from main.tscn

**Transforms (5):** identity, uniform scale, non-uniform scale, rotation, scientific notation — each from appropriate fixtures

**Node attributes (3):** instance node, script assignment, node without explicit type

**Properties (3):** custom properties (zone_name etc.), metadata, value types (Color, Vector3)

**Edge cases (3):** special node names (@CSGBox3D@...), stress count nodes (876), stress count ext_resources (68)

### 9.3 Integration Tests (`tests/test_parsers/test_tscn_integration.py`) — 6 tests

Test the query methods and full roundtrip:
- Parse all fixtures without exceptions
- find_node by path
- get_all_scripts from main.tscn
- get_all_instances from main.tscn
- get_nodes_by_group filtering
- get_nodes_by_type filtering

## 10. Implementation Sections

### Section 1: Value Parser Foundation
Create `value_parser.py` with the type detection and parsing logic. This has no dependencies on the parser state machine and can be developed and tested independently. Includes Transform3D decomposition with the YXZ Euler algorithm and determinant sign check. Includes dictionary and array parsing for nested structures.

### Section 2: Dataclass Definitions
Extend `tscn_parser.py` with SceneHeader, ExtResource, SubResource, Connection dataclasses. Extend TscnNode with all new fields. Extend TscnScene with new fields (removing old `resources`). Change parse_tscn signature to path-based.

### Section 3: State Machine Parser
Implement the line-by-line parser in `tscn_parser.py`. Section header parsing (including array and ExtResource attribute values), property line accumulation with ` = ` delimiter, multiline value accumulation via bracket depth tracking, state transitions. Wire up the value parser for property values. Use `logging` module for warnings on malformed input.

### Section 4: Tree Reconstruction & Query Methods
Implement the second pass that builds the parent-child hierarchy. Add query methods to TscnScene (find_node, get_nodes_by_group, etc.) with graceful handling of missing references.

### Section 5: Test Fixtures & Unit Tests
Create `furniture_slot_placeholder.tscn` fixture. Write all 23 unit tests against the fixture files.

### Section 6: Integration Tests & Sprint 0 Fixes
Write 6 integration tests. Update Sprint 0 test expectations for the parse_tscn signature change. Run full test suite to confirm 0 failures.

## 11. Risks and Mitigations

| Risk | Impact | Mitigation |
|------|--------|------------|
| Transform3D decomposition math errors | Silent incorrect spatial data for all tools | Extensive test cases with known-good values. Determinant sign check for negative scale. Cross-validate against SciPy during development. |
| main.tscn format version | Parser may miss section types or attributes | Read actual main.tscn header during implementation. Adapt parser if needed. |
| PackedByteArray lines (megabytes) | Regex on huge lines = slow | First-class optimization: detect packed arrays by prefix before any regex, store raw string. No regex applied to these values. |
| Multiline property values | Parser could lose data or misparse | Bracket/brace depth tracking with explicit accumulation. New section headers always break accumulation (prevent `[` in values from triggering state change). |
| Sprint 0 test breakage from API change | CI failure | Update tests immediately after API change, verify full suite before proceeding. |
| Nested dictionary/array values in sub_resources | Complex recursive parsing | Dedicated dictionary parser in value_parser.py. Recursive parse_value calls. Test with actual main.tscn _surfaces data. |
