# Wasot Syntax Testing Guide

This document describes how Wasot's syntax is tested using our markdown-based test generation framework. Tests are written in Markdown and automatically converted to native Rust tests. For the broader testing strategy, see [testing](./testing.md). For the [data model](./wasot_and_xml.md) being tested, see the data model documentation.

## Overview

Wasot uses a code generation approach where test cases are:

1. **Defined in Markdown**: Human-readable test specifications
2. **Converted to Rust**: Automatically generated as standard `#[test]` functions
3. **Run with cargo test**: Using Rust's native testing framework

This approach ensures that:
- Tests are easy to read and write in Markdown
- Tests use standard Rust tooling (no custom test runner)
- The [event-driven](./architecture.md) parser behavior is thoroughly verified
- Test failures provide detailed Rust debugging information

## Test File Locations

- **Test specifications**: `tests/language_e2e.md` (Markdown format)
- **Generated tests**: `tests/generated_syntax_tests.rs` (Auto-generated)
- **Test generator**: `scripts/generate_syntax_tests.py`
- **Test runner**: `scripts/test_syntax.sh`

## Test Format

Tests can be organized using chapters (headlines) and individual test cases:

```markdown
## Chapter Name

**test name**

Description of what this test verifies. This can be multiple paragraphs
explaining the expected behavior and any edge cases being tested.

```wasot
.example
This is the Wasot input that will be parsed.
```

```yaml
# Simple shorthand format
- block: example
- text: "This is the Wasot input that will be parsed."
- block-finished
```
```

### Test Organization
- **Chapters**: Use `## Chapter Name` to group related tests
- **Test boundaries**: Tests end at the next `**test**` or `## chapter`
- **Test names**: Bold text like `**test name**`

### Test Components
1. **Name**: Descriptive, unique test identifier
2. **Description**: Plain text explaining what's being tested
3. **Input**: Wasot code in ````wasot` blocks
4. **Expected**: Events in ````yaml` blocks

### Smart YAML Format

The test generator supports concise YAML that only checks what matters:

```yaml
# Full format (rarely needed)
- block-started:
    type: example
    level: 1
    id: myid
    
# Shorthand formats (preferred)
- block: example          # Just check block type
- text: "Some text"       # Single line of text
- text: ["Line 1", "Line 2"]  # Multiple lines
- block-finished          # No parameters needed
```

**Key principle**: Only specify values you want to test. The generator won't check unspecified fields.

## Event Format

### BlockStarted Event

```yaml
# Full format (only use what you're testing)
- block-started:
    type: "block-type"      # Block type name
    level: 2                # Nesting level (only if testing nesting)
    id: "myid"              # ID (only if testing IDs)
    inline-text: "text"     # Inline text after .. (only if present)
    args: ["arg1", "arg2"]  # Positional arguments (only if testing args)
    kwargs:                 # Keyword arguments (only if testing kwargs)
      key: "value"

# Preferred shorthand
- block: div                # Just test the block type
- block-started: paragraph  # Alternative syntax
```

### TextFinished Event

```yaml
# Full format (rarely needed)
- text-finished:
    lines: ["line 1", "line 2"]
    footnotes:              # Only if testing footnotes
      - marker: 1
        link: "http://example.com"

# Preferred shorthand
- text: "Single line"              # One line of text
- text: ["Line 1", "Line 2"]       # Multiple lines
- text-finished: "Just one line"   # Alternative syntax
```

### BlockFinished Event

```yaml
- block-finished    # Always simple, no parameters
```

## Adding New Tests

When adding new Wasot syntax features or fixing bugs, add corresponding tests:

### 1. Choose a Test Name
Pick a descriptive name that clearly indicates what's being tested:
- ✅ "Block with mixed arguments"
- ✅ "Deeply nested blocks" 
- ✅ "Quoted block type with spaces"
- ❌ "Test 1"
- ❌ "Another test"

### 2. Write the Description
Explain what the test verifies:

```markdown
**Block with mixed arguments**

Test a block that has both positional arguments and keyword arguments.
This verifies that the parser correctly separates and processes both
argument types when they appear together.
```

### 3. Create Minimal Wasot Input
Write the simplest possible Wasot code that demonstrates the feature:

```wasot
.button primary large .. disabled=true title="Click me"
Submit
```

### 4. Determine Expected Events

The easiest way to determine what events your Wasot input should generate is to use the `to-yaml-test` subcommand:

```bash
# From a file
echo '.button primary large .. disabled=true title="Click me"
Submit' > test_input.wasot
wasot to-yaml-test test_input.wasot

# From stdin
echo '.button .. primary large disabled=true title="Click me"
Submit' | wasot to-yaml-test
```

This outputs the events in exactly the same YAML format used in tests:

```yaml
- block-started:
    type: button
    level: 1
    args: ["primary", "large"]
    kwargs:
      disabled: "true"
      title: "Click me"
- text-finished:
    lines: ["Submit"]
- block-finished
```

You can then copy this output into your test specification, removing any fields you don't need to test.

### 5. Add to Test File
Insert the new test in `tests/language_e2e.md` in the appropriate chapter or create a new chapter. Tests are automatically grouped by chapter in the generated code.

## Running Tests

### Quick Start: Generate and Run Syntax Tests

```bash
./scripts/test_syntax.sh
```

This script:
1. Generates Rust tests from the Markdown specification
2. Runs only the generated syntax tests
3. Shows detailed output for any failures

### Manual Test Generation

```bash
# Generate tests from markdown
python3 scripts/generate_syntax_tests.py tests/language_e2e.md tests/generated_syntax_tests.rs

# Run the generated tests
cargo test --test generated_syntax_tests

# Run with output visible
cargo test --test generated_syntax_tests -- --nocapture

# Run a specific test by name
cargo test --test generated_syntax_tests simple_text_line -- --exact
```

### Python Dependencies

The test generator requires PyYAML:

```bash
pip3 install pyyaml
```

### Debug Test Failures

When tests fail, Rust provides detailed output:

```
---- generated_syntax_tests::basic_blocks::simple_div stdout ----
thread 'generated_syntax_tests::basic_blocks::simple_div' panicked at tests/generated_syntax_tests.rs:72:19:
assertion `left == right` failed
  left: "paragraph"
 right: "div"
```

To debug:
1. Check the test expectations in `tests/language_e2e.md`
2. Verify the YAML syntax is correct
3. Run with `--nocapture` to see all events
4. Remember: implicit text doesn't create block events

### Generate Expected YAML

Use the `to-yaml-test` subcommand to see what events are actually generated:

```bash
# Debug a failing test input
wasot to-yaml-test failing_test.wasot

# Compare with expected output in the test
diff <(wasot to-yaml-test failing_test.wasot) expected_output.yaml
```

## Test Development Workflow

### 1. Identify Test Cases
When working on Wasot parser features:
- Test the happy path (normal usage)
- Test edge cases (empty blocks, unusual nesting)
- Test error conditions (malformed syntax)
- Test interactions between features

### 2. Write Tests First (TDD)
For new features:
1. Write the test case with expected behavior
2. Run tests (they should fail)
3. Implement the feature
4. Run tests until they pass

### 3. Validate Existing Tests
When modifying parser behavior:
1. Run existing tests to see what breaks
2. Update tests if the change is intentional
3. Add new tests for the modified behavior

## Best Practices

### Test Organization
- Group related tests together in the file
- Use clear, descriptive test names
- Add tests for both common usage and edge cases
- Keep input examples minimal but complete

### Event Specifications
- Only specify fields that matter for the test
- Use optional fields (id, inline-text, args, kwargs) only when relevant
- Make expected events as specific as necessary but no more

### Test Maintenance
- Update tests when parser behavior changes intentionally
- Add regression tests when fixing bugs
- Remove obsolete tests when features are deprecated
- Keep test descriptions current with actual behavior

## Integration with CI/CD

The syntax tests integrate with standard Rust tooling:
- Generated tests run with `cargo test`
- Pre-commit hooks can regenerate tests
- CI runs the generated tests like any other Rust tests
- No custom test runner needed

To add test generation to CI:
```bash
python3 scripts/generate_syntax_tests.py tests/language_e2e.md tests/generated_syntax_tests.rs
cargo test --test generated_syntax_tests
```

## Benefits of This Approach

### Code Generation Advantages
- **Standard tooling**: Uses native Rust `#[test]` functions
- **IDE support**: Full debugging, breakpoints, and test running
- **Performance**: Compiled tests run fast
- **Maintainable**: Generated code is readable and debuggable

### Markdown Format Benefits
- **Easy to write**: Human-friendly test specifications
- **Living documentation**: Tests serve as syntax examples
- **Version control**: Markdown diffs are easy to review
- **Focused tests**: Smart defaults reduce boilerplate

## Troubleshooting

### Test Generation Issues

**PyYAML not found:**
```bash
pip3 install pyyaml
```

**Test boundaries not detected:**
- Ensure headlines use exactly `## ` (with space)
- Test names must be `**bold text**`
- Check for unclosed code blocks

### Common Test Failures

**"Expected BlockStarted for text":**
- Plain text doesn't create implicit blocks
- Only explicit `.block` syntax creates BlockStarted events

**Level mismatches:**
- Only test `level` when testing nesting behavior
- Single dot = level 1, double dot = level 2, etc.

**Missing events:**
- Check if the parser actually generates the expected events
- Use `wasot to-yaml-test file.wasot` to see actual events in test format
- Use `wasot listen-demo file.wasot` to see actual events in debug format

### YAML Syntax Tips

**Use quotes for special characters:**
```yaml
- text: "Line with: colon"
- block-started:
    type: "quoted type"
```

**Arrays can use flow or block style:**
```yaml
# Flow style
args: ["arg1", "arg2"]

# Block style
args:
  - "arg1"
  - "arg2"
```