# Test Implementation: Consolidate Nested Dictionary Operations

## Issue

- **Issue Number**: #604
- **Title**: Consolidate nested dictionary helper functions into
  `lionagi.utils.nested`
- **Link**: [Issue #604](https://github.com/khive-ai/lionagi/issues/604)

## Overview

This document outlines the testing strategy for the consolidation of nested
dictionary operations into a single module (`lionagi.utils.nested`). The
implementation will consolidate functionality from `lionagi/libs/nested` and
potentially other utility modules into a single, well-defined module.

## Test Strategy

The testing strategy will focus on ensuring that the consolidated module
maintains the same functionality as the existing implementations while providing
a more consistent and maintainable API.

### Test Categories

1. **Functional Tests**: Verify that each function in the consolidated module
   behaves as expected
2. **Compatibility Tests**: Ensure backward compatibility with existing code
3. **Edge Case Tests**: Test handling of edge cases (empty structures, nested
   lists, mixed types)
4. **Performance Tests**: Compare performance of consolidated implementation
   with existing implementation

### Test Environment

- Python 3.10+ (matching project requirements)
- pytest for test execution
- pytest-cov for coverage reporting

## Test Cases

### 1. Functional Tests

#### 1.1 flatten/unflatten

```python
def test_flatten_basic():
    """Test basic flattening of nested dictionaries."""
    nested = {"a": 1, "b": {"c": 2, "d": [3, 4]}}
    expected = {'a': 1, 'b|c': 2, 'b|d|0': 3, 'b|d|1': 4}
    assert flatten(nested) == expected

def test_flatten_with_max_depth():
    """Test flattening with max_depth parameter."""
    nested = {"a": 1, "b": {"c": 2, "d": {"e": 3}}}
    expected = {'a': 1, 'b|c': 2, 'b|d': {"e": 3}}
    assert flatten(nested, max_depth=2) == expected

def test_unflatten_basic():
    """Test basic unflattening of flat dictionaries."""
    flat = {'a': 1, 'b|c': 2, 'b|d|0': 3, 'b|d|1': 4}
    expected = {"a": 1, "b": {"c": 2, "d": [3, 4]}}
    assert unflatten(flat) == expected
```

#### 1.2 nget/nset/npop

```python
def test_nget_basic():
    """Test basic nested get operation."""
    nested = {"a": {"b": {"c": 3}}}
    assert nget(nested, ["a", "b", "c"]) == 3
    assert nget(nested, ["a", "b"]) == {"c": 3}

def test_nget_with_default():
    """Test nested get with default value."""
    nested = {"a": {"b": 2}}
    assert nget(nested, ["a", "c"], default=10) == 10

def test_nset_basic():
    """Test basic nested set operation."""
    nested = {"a": {"b": 2}}
    nset(nested, ["a", "c"], 3)
    assert nested == {"a": {"b": 2, "c": 3}}

def test_npop_basic():
    """Test basic nested pop operation."""
    nested = {"a": {"b": 2, "c": 3}}
    result = npop(nested, ["a", "c"])
    assert result == 3
    assert nested == {"a": {"b": 2}}
```

#### 1.3 ninsert/nmerge/nfilter

```python
def test_ninsert_basic():
    """Test basic nested insert operation."""
    nested = {"a": [1, 2]}
    ninsert(nested, ["a", 1], 3)
    assert nested == {"a": [1, 3, 2]}

def test_nmerge_basic():
    """Test basic nested merge operation."""
    nested1 = {"a": {"b": 1}}
    nested2 = {"a": {"c": 2}}
    result = nmerge(nested1, nested2)
    assert result == {"a": {"b": 1, "c": 2}}

def test_nfilter_basic():
    """Test basic nested filter operation."""
    nested = {"a": 1, "b": {"c": 2, "d": 3}}
    result = nfilter(nested, lambda p, k, v: isinstance(v, int) and v > 1)
    assert result == {"b": {"c": 2, "d": 3}}
```

### 2. Compatibility Tests

```python
def test_backward_compatibility_imports():
    """Test that old import paths still work."""
    from lionagi.libs.nested.flatten import flatten as old_flatten
    from lionagi.utils.nested import flatten as new_flatten
    
    nested = {"a": 1, "b": {"c": 2}}
    assert old_flatten(nested) == new_flatten(nested)

def test_backward_compatibility_behavior():
    """Test that behavior is consistent with old implementation."""
    # Import both old and new implementations
    from lionagi.libs.nested.nget import nget as old_nget
    from lionagi.utils.nested import nget as new_nget
    
    nested = {"a": {"b": 2}}
    assert old_nget(nested, ["a", "b"]) == new_nget(nested, ["a", "b"])
```

### 3. Edge Case Tests

```python
def test_empty_structures():
    """Test handling of empty structures."""
    assert flatten({}) == {}
    assert unflatten({}) == {}
    
    # Test with empty nested structures
    nested = {"a": {}, "b": []}
    flat = flatten(nested)
    assert unflatten(flat) == nested

def test_nested_lists():
    """Test handling of nested lists."""
    nested = [1, [2, [3, 4]]]
    flat = flatten(nested, coerce_sequence="dict")
    assert flat == {'0': 1, '1|0': 2, '1|1|0': 3, '1|1|1': 4}
    assert unflatten(flat) == nested

def test_mixed_types():
    """Test handling of mixed types."""
    nested = {"a": 1, "b": [2, {"c": 3}]}
    flat = flatten(nested)
    assert unflatten(flat) == nested
```

### 4. Performance Tests

```python
def test_performance_flatten():
    """Test performance of flatten operation."""
    import time
    from lionagi.libs.nested.flatten import flatten as old_flatten
    from lionagi.utils.nested import flatten as new_flatten
    
    # Create a deeply nested structure
    nested = {}
    current = nested
    for i in range(100):
        current[f"level{i}"] = {}
        current = current[f"level{i}"]
    current["value"] = "test"
    
    # Measure old implementation
    start = time.time()
    old_result = old_flatten(nested)
    old_time = time.time() - start
    
    # Measure new implementation
    start = time.time()
    new_result = new_flatten(nested)
    new_time = time.time() - start
    
    # Assert results are the same
    assert old_result == new_result
    
    # Print performance comparison (optional)
    print(f"Old implementation: {old_time:.6f}s")
    print(f"New implementation: {new_time:.6f}s")
    print(f"Speedup: {old_time/new_time:.2f}x")
```

## Coverage Goals

- Aim for 100% code coverage of the consolidated module
- Ensure all edge cases are covered
- Test all parameter combinations for each function

## Test Execution

Tests will be executed using pytest:

```bash
uv run pytest tests/utils/test_nested.py -v
```

Coverage will be measured using pytest-cov:

```bash
uv run pytest tests/utils/test_nested.py --cov=lionagi.utils.nested
```

## Continuous Integration

Tests will be integrated into the existing CI pipeline to ensure that changes to
the consolidated module do not break existing functionality.
