# Void Element Implementation - Todo List

This document provides a detailed, milestone-based implementation plan for adding void elements to wasot. Each milestone ends with a compiling project and verifiable tests.

## Syntax Reference

```wasot
.hr:                                    # Basic void element
.img: alt text here                     # Void element with inline text
.img: alt text .. src="photo.jpg"       # Void element with args
.img: alt text .. src="x.jpg" :img1     # Full void element with ID
..nested-void:                          # Nested void element (level 2)
```

---

## Milestone 1: Core Data Model (Parser Layer)

**Goal**: Add `is_void` field to `MetaLine` struct and parse trailing `:` in type names.

### Tasks

- [x] **1.1** Add `is_void: bool` field to `MetaLine` struct
  - File: `src/parsers/wasot/parser.rs` (line ~102)
  - Add field with `#[serde(default)]` for backwards compatibility
  - Update `MetaLine::new()` if it exists

- [x] **1.2** Update `extract_line_type()` to detect and strip trailing `:`
  - File: `src/parsers/wasot/parser.rs` (line ~548)
  - Return tuple `(type_name, remaining, is_void)` or modify signature
  - Strip `:` from type_name before returning

- [x] **1.3** Update `MetaLine::parse()` to set `is_void` flag
  - File: `src/parsers/wasot/parser.rs` (line ~262)
  - Pass through `is_void` from `extract_line_type()`

- [x] **1.4** Update all MetaLine constructors in tests
  - Files: All test files that create `MetaLine` instances
  - Add `is_void: false` to existing test constructors

- [x] **1.5** Add parser unit tests for void element detection
  - File: `src/parsers/wasot/parser.rs` (tests module)
  - Test cases:
    - [x] `.hr:` -> `type_name="hr", is_void=true`
    - [x] `.hr` -> `type_name="hr", is_void=false`
    - [x] `.img: alt text` -> `type_name="img", is_void=true, inline_text="alt text"`
    - [x] `.img: alt .. src="x.jpg"` -> `type_name="img", is_void=true, args=["src=x.jpg"]`
    - [x] `.img: alt .. src="x.jpg" :id1` -> with ID
    - [x] `..nested:` -> `type_name="nested", level=2, is_void=true`
    - [x] `."quoted type":` -> `type_name="quoted type", is_void=true`

**Verification**: `cargo test` passes, project compiles - **DONE**

---

## Milestone 2: Event System

**Goal**: Add `VoidElement` event type and update event validation.

### Tasks

- [x] **2.1** Add `VoidElement` variant to `Event` enum
  - File: `src/events/events.rs` (line ~56)
  ```rust
  VoidElement {
      meta_line: MetaLine,
  },
  ```

- [x] **2.2** Update `Event::validate()` method
  - File: `src/events/events.rs` (line ~83)
  - Add validation for `VoidElement`:
    - Check `meta_line.level <= 100`
    - Check `meta_line.is_void == true`

- [x] **2.3** Add event unit tests
  - File: `src/events/events.rs` (tests module)
  - Test cases:
    - [x] `VoidElement` event validates successfully
    - [x] `VoidElement` event can be cloned
    - [x] `VoidElement` event can be compared with PartialEq
    - [x] `VoidElement` event Debug formatting works

**Verification**: `cargo test` passes, project compiles - **DONE**

---

## Milestone 3: Element Model

**Goal**: Add `VoidElement` variant to the `Element` enum.

### Tasks

- [x] **3.1** Add `VoidElement(MetaLine)` variant to `Element` enum
  - File: `src/parsers/wasot/block.rs` (line ~6)

- [x] **3.2** Update `Element::is_empty()` method
  - File: `src/parsers/wasot/block.rs` (line ~86)
  - `VoidElement` is never empty (returns `false`)

- [x] **3.3** Update `Element::as_block()` method
  - File: `src/parsers/wasot/block.rs` (line ~73)
  - Return `None` for `VoidElement`

- [x] **3.4** Update `Element::as_text()` method
  - File: `src/parsers/wasot/block.rs` (line ~82)
  - Return `None` for `VoidElement`

- [x] **3.5** Update `Element::as_context_line()` method
  - File: `src/parsers/wasot/block.rs` (line ~115)
  - Return `None` for `VoidElement`

- [x] **3.6** Add `Element::as_void_element()` method
  - File: `src/parsers/wasot/block.rs`
  - Return `Some(&MetaLine)` for `VoidElement`, `None` otherwise

- [x] **3.7** Add element unit tests
  - File: `src/parsers/wasot/block.rs` (tests module)
  - Test cases:
    - [x] `VoidElement` variant `is_empty()` returns `false`
    - [x] `VoidElement` variant `as_void_element()` returns `Some`
    - [x] `VoidElement` variant `as_block()` returns `None`
    - [x] `VoidElement` serialization/deserialization works

**Verification**: `cargo test` passes, project compiles - **DONE**

---

## Milestone 4: Emitter (Event Producer)

**Goal**: Update `WasotFileEmitter` to emit `VoidElement` events.

### Tasks

- [x] **4.1** Update `process_line_with_events()` in `WasotFileEmitter`
  - File: `src/emitters/wasot_file_emitter.rs` (line ~134)
  - After parsing metaline, check `is_void` flag
  - If `is_void`, emit `VoidElement` event instead of `BlockStarted`
  - Do NOT push void elements onto the block stack

- [x] **4.2** Ensure void elements don't affect block nesting
  - File: `src/emitters/wasot_file_emitter.rs`
  - Void elements should not be pushed to the element parser stack
  - Subsequent blocks should nest correctly after void elements

- [x] **4.3** Add emitter unit tests
  - File: `src/emitters/wasot_file_emitter.rs` (tests module)
  - Test cases:
    - [x] `.hr:` emits `VoidElement` event
    - [x] `.hr:` does NOT emit `BlockFinished`
    - [x] Void element followed by regular block nests correctly
    - [x] Void element inside a block works correctly
    - [x] Multiple consecutive void elements work

**Verification**: `cargo test` passes, project compiles - **DONE**

---

## Milestone 5: WasotReconstructor (Pretty Printing)

**Goal**: Reconstruct void elements with trailing `:` syntax.

### Tasks

- [x] **5.1** Handle `VoidElement` event in `on_element_event()`
  - File: `src/consumers/wasot_reconstructor.rs` (line ~253)
  - Output the metaline with trailing `:` after type name

- [x] **5.2** Add `format_void_meta_line()` method
  - File: `src/consumers/wasot_reconstructor.rs`
  - Similar to `format_meta_line()` but adds `:` after type name
  - Handle quoted type names: `."quoted":`

- [x] **5.3** Add reconstructor unit tests
  - File: `src/consumers/wasot_reconstructor.rs` (tests module)
  - Test cases:
    - [x] Basic void element: `.hr:` round-trips correctly
    - [x] Void element with inline text round-trips
    - [x] Void element with args round-trips
    - [x] Void element with ID round-trips
    - [x] Nested void element (level 2) round-trips
    - [x] Quoted type void element round-trips

**Verification**: `cargo test` passes, project compiles - **DONE**

---

## Milestone 6: XML Consumer

**Goal**: Output void elements as self-closing XML tags.

### Tasks

- [x] **6.1** Handle `VoidElement` event in `process_event()`
  - File: `src/consumers/to_xml_consumer.rs` (line ~339)
  - Create a self-closing `<block />` element with `void="true"` attribute

- [x] **6.2** Add `write_void_element()` method
  - File: `src/consumers/to_xml_consumer.rs`
  - Similar to `start_element()` but writes self-closing tag
  - Include `void="true"` attribute to distinguish from empty blocks

- [x] **6.3** Add XML consumer unit tests
  - File: `src/consumers/to_xml_consumer.rs` (tests module)
  - Test cases:
    - [x] Void element produces `<block ... void="true" />`
    - [x] Void element has correct type attribute
    - [x] Void element has correct level attribute
    - [x] Void element with ID has id attribute
    - [x] Void element inline text becomes `<inline-text>` child
    - [x] Void element args become `<arg>` children

**Verification**: `cargo test` passes, project compiles - **DONE**

---

## Milestone 7: JSONL Consumer

**Goal**: Output void elements in JSONL format.

### Tasks

- [x] **7.1** Handle `VoidElement` event in `process_event()`
  - File: `src/consumers/to_jsonl_consumer.rs` (line ~40)
  - Level 1 void elements are written directly as root elements
  - Include `is_void: true` in JSON output

- [x] **7.2** Update `StreamingJsonExporter` similarly
  - File: `src/consumers/to_jsonl_consumer.rs` (line ~125)
  - Same handling as `ToJsonlConsumer`

- [x] **7.3** Add JSONL consumer unit tests
  - File: `src/consumers/to_jsonl_consumer.rs` (tests module)
  - Test cases:
    - [x] Root-level void element is written to output
    - [x] Nested void element is NOT written (only level 1)
    - [x] JSON includes `is_void: true` field
    - [x] Void element JSON has all metaline fields

**Verification**: `cargo test` passes, project compiles - **DONE**

---

## Milestone 8: XML Listener (Stdout)

**Goal**: Update the stdout XML listener to handle void elements.

### Tasks

- [x] **8.1** Handle `VoidElement` event in `handle()` method
  - File: `src/consumers/xml_listener.rs`
  - Write self-closing block element

- [x] **8.2** Add XML listener tests
  - File: `src/consumers/xml_listener.rs` (tests module)
  - Test case: void element outputs correctly

**Verification**: `cargo test` passes, project compiles - **DONE**

---

## Milestone 9: Binary Protobuf Consumer

**Goal**: Support void elements in binary format.

### Tasks

- [x] **9.1** Handle `VoidElement` event in `convert_and_write()`
  - File: `src/consumers/binary_protobuf_consumer.rs`
  - Serialize void element similar to `BlockStarted` but with void flag

- [x] **9.2** Add binary consumer tests
  - File: `src/consumers/binary_protobuf_consumer.rs` (tests module)
  - Test case: void element serializes correctly

**Verification**: `cargo test` passes, project compiles - **DONE**

---

## Milestone 10: SQL File Consumer

**Goal**: Support void elements in SQL output.

### Tasks

- [x] **10.1** Handle `VoidElement` event in `process_event()`
  - File: `src/consumers/sql_file_consumer.rs`
  - Insert void element as block with `is_void=1` column

- [x] **10.2** Add SQL consumer tests
  - File: `src/consumers/sql_file_consumer.rs` (tests module)
  - Test case: void element generates correct SQL

**Verification**: `cargo test` passes, project compiles - **DONE**

---

## Milestone 11: SQLite Consumer

**Goal**: Support void elements in SQLite database storage.

### Tasks

- [x] **11.1** Handle `VoidElement` event in SQLite consumer
  - File: `src/consumers/sqlite_consumer.rs`
  - Store void element with `is_void` flag in blocks table

- [x] **11.2** Add SQLite consumer tests
  - File: `src/consumers/sqlite_consumer.rs` (tests module)
  - Test case: void element stored correctly in database

**Verification**: `cargo test` passes, project compiles - **DONE**

---

## Milestone 12: Selector Consumer

**Goal**: Support void elements in CSS-like selector matching.

### Tasks

- [x] **12.1** Handle `VoidElement` event in selector consumer
  - File: `src/consumers/selector_consumer.rs` (line ~48)
  - Void elements should be matchable by selectors
  - Pass through matching void elements

- [x] **12.2** Update `StackFrameCollector` if needed
  - File: `src/parsers/selectors/stack_frame_collector.rs`
  - Void elements may need special handling (they don't push/pop stack)

- [x] **12.3** Add selector consumer tests
  - File: `src/consumers/selector_consumer.rs` (tests module)
  - Test cases:
    - [x] Void element matches type selector (e.g., `hr`)
    - [x] Void element matches attribute selector
    - [x] Void element doesn't affect descendant matching

**Verification**: `cargo test` passes, project compiles - **DONE**

---

## Milestone 13: Protobuf Binary Emitter

**Goal**: Support void elements when reading from binary format.

### Tasks

- [x] **13.1** Emit `VoidElement` when reading void elements from binary
  - File: `src/emitters/protobuf_binary_emitter.rs`
  - Detect void flag and emit appropriate event

- [x] **13.2** Add emitter tests
  - Test case: binary void element emits `VoidElement` event

**Verification**: `cargo test` passes, project compiles - **DONE**

---

## Milestone 14: JSONL Emitter

**Goal**: Support void elements when reading from JSONL format.

### Tasks

- [x] **14.1** Emit `VoidElement` when reading void elements from JSONL
  - File: `src/emitters/jsonl_emitter.rs`
  - Detect `is_void: true` and emit `VoidElement` event

- [x] **14.2** Add emitter tests
  - Test case: JSONL void element emits `VoidElement` event

**Verification**: `cargo test` passes, project compiles - **DONE**

---

## Milestone 15: SQL File Emitter

**Goal**: Support void elements when reading from SQL format.

### Tasks

- [x] **15.1** Emit `VoidElement` when reading void elements from SQL
  - File: `src/emitters/sql_file_emitter.rs`
  - Detect `is_void=1` and emit `VoidElement` event

- [x] **15.2** Add emitter tests
  - Test case: SQL void element emits `VoidElement` event

**Verification**: `cargo test` passes, project compiles - **DONE**

---

## Milestone 16: SQLite Emitter

**Goal**: Support void elements when reading from SQLite database.

### Tasks

- [x] **16.1** Emit `VoidElement` when reading void elements from SQLite
  - File: `src/emitters/sqlite_emitter.rs`
  - Detect `is_void` flag and emit `VoidElement` event

- [x] **16.2** Add emitter tests
  - Test case: SQLite void element emits `VoidElement` event

**Verification**: `cargo test` passes, project compiles - **DONE**

---

## Milestone 17: Other Consumers

**Goal**: Update remaining consumers to handle void elements.

### Tasks

- [x] **17.1** Update `action_consumer.rs`
  - Handle `VoidElement` event appropriately

- [x] **17.2** Update `links_printer.rs`
  - Handle `VoidElement` event (pass through or ignore)

- [x] **17.3** Update `to_yaml_test_consumer.rs`
  - Handle `VoidElement` event

- [x] **17.4** Update any other consumers found
  - Check `src/consumers/mod.rs` for complete list

**Verification**: `cargo test` passes, project compiles - **DONE**

---

## Milestone 18: Integration Tests

**Goal**: Add comprehensive integration tests.

### Tasks

- [ ] **18.1** Add roundtrip tests
  - File: `tests/roundtrip_tests.rs`
  - Test cases:
    - [ ] Wasot -> Wasot roundtrip preserves void elements
    - [ ] Wasot -> XML -> Wasot roundtrip (if applicable)
    - [ ] Wasot -> JSONL -> Wasot roundtrip
    - [ ] Wasot -> Binary -> Wasot roundtrip
    - [ ] Wasot -> SQLite -> Wasot roundtrip
    - [ ] Wasot -> SQL -> Wasot roundtrip

- [ ] **18.2** Add CLI tests
  - File: `tests/cli_tests.rs`
  - Test cases:
    - [ ] `wasot file.wasot` pretty-prints void elements correctly
    - [ ] `wasot file.wasot -t xml` converts void elements
    - [ ] `wasot file.wasot -t jsonl` converts void elements
    - [ ] `wasot file.wasot --select "hr"` matches void elements

- [ ] **18.3** Add syntax tests
  - File: `tests/generated_syntax_tests.rs` or examples
  - Test cases for various void element syntaxes

**Verification**: `cargo test` passes, `cargo test -- --ignored` passes

---

## Milestone 19: Documentation

**Goal**: Update documentation to describe void elements.

### Tasks

- [ ] **19.1** Update `docs/ast.md`
  - Add `VoidElement` to the AST definition pseudocode

- [ ] **19.2** Update `docs/wasot_and_xml.md`
  - Add void element examples
  - Update data model diagram

- [ ] **19.3** Update `docs/architecture.md`
  - Mention `VoidElement` event type

- [ ] **19.4** Update `README.md` if needed
  - Add void element syntax to quick reference

- [ ] **19.5** Add void element examples
  - File: `examples/void_elements.wasot`
  - Show various void element use cases

**Verification**: Documentation is accurate and consistent

---

## Milestone 20: Final Verification

**Goal**: Ensure 100% coverage and all tests pass.

### Tasks

- [x] **20.1** Run full test suite
  ```bash
  cargo test
  cargo test -- --ignored
  ./scripts/test_all.sh
  ```

- [x] **20.2** Run pre-commit checks
  ```bash
  ./scripts/dev/pre-commit.sh
  ```

- [x] **20.3** Verify code coverage for new features
  - All new code paths should be tested
  - Check with `cargo tarpaulin` if available

- [x] **20.4** Manual testing of CLI
  - Test all conversion paths with void elements
  - Test selector matching with void elements

- [x] **20.5** Review for any missed consumers/emitters
  - Check `src/consumers/mod.rs`
  - Check `src/emitters/mod.rs`
  - Check all match statements on `Event` enum

**Verification**: All tests pass, pre-commit succeeds, no compiler warnings - **DONE**

---

## Summary of Files to Modify

### Core Parser
- `src/parsers/wasot/parser.rs` - MetaLine struct, extract_line_type(), parse() - **DONE**

### Events
- `src/events/events.rs` - Event enum, validate() - **DONE**

### Element Model
- `src/parsers/wasot/block.rs` - Element enum - **DONE**

### Emitters (Producers)
- `src/emitters/wasot_file_emitter.rs` - Main wasot emitter - **DONE**
- `src/emitters/protobuf_binary_emitter.rs` - Binary format emitter - **DONE**
- `src/emitters/jsonl_emitter.rs` - JSONL format emitter - **DONE**
- `src/emitters/sql_file_emitter.rs` - SQL format emitter - **DONE**
- `src/emitters/sqlite_emitter.rs` - SQLite emitter - **DONE**

### Consumers
- `src/consumers/wasot_reconstructor.rs` - Pretty printing - **DONE**
- `src/consumers/to_xml_consumer.rs` - XML output - **DONE**
- `src/consumers/xml_listener.rs` - XML stdout - **DONE**
- `src/consumers/to_jsonl_consumer.rs` - JSONL output - **DONE**
- `src/consumers/binary_protobuf_consumer.rs` - Binary output - **DONE**
- `src/consumers/sql_file_consumer.rs` - SQL output - **DONE**
- `src/consumers/sqlite_consumer.rs` - SQLite output - **DONE**
- `src/consumers/selector_consumer.rs` - Selector filtering - **DONE**
- `src/consumers/action_consumer.rs` - Action handling - **DONE**
- `src/consumers/links_printer.rs` - Link printing - **DONE**
- `src/consumers/to_yaml_test_consumer.rs` - YAML test output - **DONE**
- `src/consumers/protobuf_consumer.rs` - Protobuf consumer - **DONE**

### Selectors
- `src/parsers/selectors/stack_frame_collector.rs` - Selector stack - **DONE**

### Tests
- `tests/roundtrip_tests.rs` - TODO
- `tests/cli_tests.rs` - TODO
- `tests/generated_syntax_tests.rs` - TODO

### Documentation
- `docs/ast.md` - TODO
- `docs/wasot_and_xml.md` - TODO
- `docs/architecture.md` - TODO
- `README.md` - TODO
- `examples/void_elements.wasot` - TODO

---

## Estimated Effort by Milestone

| Milestone | Effort | Risk | Status |
|-----------|--------|------|--------|
| 1. Core Data Model | Medium | Low | **DONE** |
| 2. Event System | Low | Low | **DONE** |
| 3. Element Model | Low | Low | **DONE** |
| 4. Emitter | Medium | Medium | **DONE** |
| 5. Reconstructor | Medium | Low | **DONE** |
| 6. XML Consumer | Low | Low | **DONE** |
| 7. JSONL Consumer | Low | Low | **DONE** |
| 8-17. Other Consumers/Emitters | Medium | Low | **DONE** |
| 18. Integration Tests | High | Low | TODO |
| 19. Documentation | Low | Low | TODO |
| 20. Final Verification | Medium | Low | **DONE** |

---

## Dependencies Between Milestones

```
1 (Parser) ─┬─> 2 (Events) ───> 4 (Emitter) ───> 5-17 (Consumers)
            │                                          │
            └─> 3 (Element) ───────────────────────────┘
                                                       │
                                                       v
                                               18 (Integration)
                                                       │
                                                       v
                                               19 (Docs) ──> 20 (Final)
```

Milestones 1, 2, 3 must be done first. Then 4. Then 5-17 can be done in parallel.


## Write docs

in docs/ about the void elements and link thhis
https://html.spec.whatwg.org/multipage/syntax.html#void-elements
