# Strict Validation Implementation Plan

## Goal

Implement strict syntax validation to match Neo4j behavior, making nxCypher TCK-compliant by default, with optional lenient mode.

## Current Problem

**88 XPASS tests** = Tests expecting failure (SyntaxError) but passing due to lenient parser.

## Solution Architecture

### 1. Add Strict Mode Flag

**Location**: `nxcypher/__init__.py` and `nxcypher/interpreter.py`

```python
class NXCypher:
    def __init__(self, graph, strict=True):
        self.strict = strict
        # Pass to interpreter

class NXCypherInterpreter:
    def __init__(self, strict=True):
        self.strict = strict
```

**Default**: `strict=True` (Neo4j-compatible)
**Opt-out**: `strict=False` (current lenient behavior)

### 2. Validation Categories

Based on 88 XPASS tests, implement these validation checks:

#### A. Variable Binding Validation (26 tests)
**Files**: clauses::create (10), clauses::match (16)

**Errors to detect**:
- `VariableAlreadyBound` - Variable used with different labels/properties
- `VariableTypeConflict` - Variable used as both node and relationship

**Examples**:
```cypher
CREATE (n:Foo), (n:Bar)           → VariableAlreadyBound
MATCH ()-[r]-() MATCH (r)         → VariableTypeConflict
CREATE (n {}), (n:Label)          → VariableAlreadyBound
```

**Implementation**: Track variable bindings in interpreter, validate consistency

#### B. Aggregation Validation (5 tests)
**Files**: clauses::return (5)

**Errors to detect**:
- `NestedAggregation` - Aggregations inside aggregations
- `NonConstantExpression` - Non-deterministic functions in aggregations

**Examples**:
```cypher
RETURN count(count(*))            → NestedAggregation
RETURN count(rand())              → NonConstantExpression
```

**Implementation**: Check expression trees for nested aggregations

#### C. CALL Validation (15 tests)
**Files**: clauses::call (15)

**Errors to detect**:
- `InvalidNumberOfArguments` - Wrong argument count
- `UnknownProcedure` - Procedure doesn't exist
- `VariableAlreadyBound` - YIELD shadows existing variable

**Examples**:
```cypher
CALL proc(arg1)                   → InvalidNumberOfArguments (needs 2)
CALL unknown.proc()               → UnknownProcedure
CALL proc() YIELD n MATCH (n)     → VariableAlreadyBound
```

**Implementation**: Since CALL not supported, always raise error in strict mode

#### D. Literal Validation (14 tests)
**Files**: expressions::literals (14)

**Errors to detect**:
- `UnexpectedSyntax` - Invalid literal format
- `InvalidMapKey` - Map keys with invalid characters

**Examples**:
```cypher
RETURN {k1#k: 1}                  → UnexpectedSyntax (# in key)
RETURN {k1.k: 1}                  → UnexpectedSyntax (. in key)
```

**Implementation**: Add validation in literal parsing

#### E. Other Validations (28 tests)
**Files**: Various (with-orderBy, boolean, quantifier, etc.)

**Errors to detect**:
- Various syntax and semantic errors
- Will implement based on specific test failures

## Implementation Phases

### Phase 1: Infrastructure (2 hours)

**File**: `nxcypher/__init__.py`

1. Add `strict` parameter to `NXCypher.__init__`
2. Pass `strict` to interpreter
3. Create validation exception classes:
   ```python
   class CypherValidationError(Exception):
       pass

   class VariableAlreadyBound(CypherValidationError):
       pass

   class VariableTypeConflict(CypherValidationError):
       pass

   class NestedAggregation(CypherValidationError):
       pass
   # ... etc
   ```

4. Update test_tck.py to use strict mode by default:
   ```python
   nxc = NXCypher(graph, strict=True)
   ```

### Phase 2: Variable Binding Validation (4-6 hours)

**Target**: Fix 26 XPASS tests (create + match)

**Implementation**:

1. **Track variable bindings** in interpreter:
   ```python
   class BindingTracker:
       def __init__(self):
           self.bindings = {}  # var_name -> (type, labels, properties)

       def bind(self, var_name, var_type, labels=None, properties=None):
           if var_name in self.bindings:
               existing = self.bindings[var_name]
               if existing['type'] != var_type:
                   raise VariableTypeConflict(...)
               if labels and existing['labels'] != labels:
                   raise VariableAlreadyBound(...)
           self.bindings[var_name] = {'type': var_type, 'labels': labels, ...}
   ```

2. **Validate in CREATE**:
   - Check variable reuse with different labels
   - Check variable reuse with different properties

3. **Validate in MATCH**:
   - Check relationship variable used as node
   - Check node variable used as relationship

**Test**: Run Create1, Create2, Match1 tests → Should move from XPASS to XFAIL

### Phase 3: Aggregation Validation (2-3 hours)

**Target**: Fix 5 XPASS tests (return)

**Implementation**:

1. **Detect nested aggregations**:
   ```python
   def _contains_nested_aggregation(node):
       if is_aggregation(node):
           for child in node.children:
               if contains_aggregation(child):
                   raise NestedAggregation(...)
   ```

2. **Detect non-constant in aggregations**:
   ```python
   def _validate_aggregation_args(node):
       # Check for rand(), randomUUID(), etc.
       if contains_nondeterministic_function(node):
           raise NonConstantExpression(...)
   ```

**Test**: Run Return6 tests → Should move from XPASS to XFAIL

### Phase 4: CALL Validation (1 hour)

**Target**: Fix 15 XPASS tests (call)

**Implementation**:

Since CALL is not supported, in strict mode:
```python
def _handle_standalone_call(self, node):
    if self.strict:
        raise UnknownProcedure("CALL clause not supported")
    else:
        # Current lenient behavior (NotImplementedError)
        raise NotImplementedError(...)
```

**Test**: Run Call1, Call2, Call5 tests → Should move from XPASS to XFAIL

### Phase 5: Literal Validation (3-4 hours)

**Target**: Fix 14 XPASS tests (literals)

**Implementation**:

1. **Validate map keys**:
   ```python
   def _validate_map_key(key):
       if self.strict and not is_valid_identifier(key):
           raise UnexpectedSyntax(f"Invalid map key: {key}")
   ```

2. **Validate literal formats**:
   - Check for invalid characters
   - Check for malformed syntax

**Test**: Run Literals tests → Should move from XPASS to XFAIL

### Phase 6: Remaining Validations (4-6 hours)

**Target**: Fix remaining 28 XPASS tests

**Implementation**: Add specific validations as tests reveal issues

## Expected Results

### Before Implementation
```
619 PASSED   (38.3%)
 88 XPASSED  ( 5.4%) ← Unexpected passes
161 FAILED   (10.0%)
747 XFAILED  (46.3%)
```

### After Implementation (Strict Mode)
```
619 PASSED   (38.3%) ← Unchanged
  0 XPASSED  ( 0.0%) ← Fixed! All converted to XFAIL
161 FAILED   (10.0%) ← Unchanged
835 XFAILED  (51.7%) ← Increased (88 moved from XPASS)
```

### With Lenient Mode
```python
nxc = NXCypher(graph, strict=False)
```
Results should match current behavior (88 XPASS)

## Testing Strategy

### Unit Tests
Add tests for each validation:
```python
def test_variable_already_bound_strict():
    nxc = NXCypher(G, strict=True)
    with pytest.raises(VariableAlreadyBound):
        nxc.run("CREATE (n:Foo), (n:Bar)")

def test_variable_already_bound_lenient():
    nxc = NXCypher(G, strict=False)
    # Should not raise
    result = nxc.run("CREATE (n:Foo), (n:Bar)")
```

### TCK Tests
1. Run with `strict=True` (default) → 88 XPASS should become XFAIL
2. Run with `strict=False` → Should maintain current behavior

### Regression Tests
Ensure existing 619 PASSED tests still pass in strict mode

## Estimated Effort

| Phase | Hours | Tests Fixed |
|-------|-------|-------------|
| 1. Infrastructure | 2 | 0 |
| 2. Variable Binding | 4-6 | 26 |
| 3. Aggregation | 2-3 | 5 |
| 4. CALL | 1 | 15 |
| 5. Literals | 3-4 | 14 |
| 6. Remaining | 4-6 | 28 |
| **Total** | **16-22 hours** | **88** |

## Success Criteria

1. ✅ All 88 XPASS tests convert to XFAIL in strict mode
2. ✅ All 619 PASSED tests remain PASSED in strict mode
3. ✅ Lenient mode (`strict=False`) maintains backward compatibility
4. ✅ Clear error messages for validation failures
5. ✅ Documentation updated explaining strict vs lenient modes

## Rollout Plan

1. **Implement** validation in phases
2. **Test** each phase before moving to next
3. **Document** strict mode in user guide
4. **Default** to strict mode for TCK compliance
5. **Provide** migration guide for users needing lenient mode

## Documentation Needed

1. **User Guide**: Explain strict vs lenient modes
2. **Migration Guide**: How to enable lenient mode if needed
3. **Error Reference**: Document all validation errors
4. **TCK Report**: Update to show 0 XPASS with strict mode

---

**Recommendation**: Implement this plan to achieve proper TCK compliance while maintaining backward compatibility through optional lenient mode.
