# Diagnosis and Solution Architecture

## Problem Statement

The `ParquetAdapter` is missing the `read_transfer_timestamps()` method, causing an `AttributeError` when running pattern detection. This method exists in `ClickHouseAdapter` but was never implemented for the Parquet-based workflow.

## Root Cause Analysis

### 1. Missing Method Implementation
**Location**: [`src/chainswarm_analyzers_baseline/adapters/parquet.py`](src/chainswarm_analyzers_baseline/adapters/parquet.py)

The `ParquetAdapter` class has these read methods:
- ✅ `read_transfers()` - Lines 40-63
- ✅ `read_money_flows()` - Lines 65-92  
- ✅ `read_assets()` - Lines 94-112
- ✅ `read_asset_prices()` - Lines 114-143
- ✅ `read_address_labels()` - Lines 145-177
- ❌ `read_transfer_timestamps()` - **MISSING**

### 2. ClickHouseAdapter Reference Implementation
**Location**: [`src/chainswarm_analyzers_baseline/adapters/clickhouse.py:108-151`](src/chainswarm_analyzers_baseline/adapters/clickhouse.py:108)

```python
def read_transfer_timestamps(
    self,
    start_timestamp_ms: int,
    end_timestamp_ms: int,
    addresses: List[str]
) -> Dict[str, List[Dict[str, Any]]]:
```

**Returns**: Dictionary mapping each address to a list of timestamp records:
```python
{
    "address1": [
        {"timestamp": 1767735224000, "volume": 100.5, "counterparty": "address2"},
        {"timestamp": 1767735225000, "volume": 50.0, "counterparty": "address3"},
    ],
    "address2": [...],
}
```

### 3. Usage in Pipeline
**Location**: [`src/chainswarm_analyzers_baseline/pipeline/baseline.py:73-75`](src/chainswarm_analyzers_baseline/pipeline/baseline.py:73)

```python
if run_patterns:
    transfers = self.adapter.read_transfers(start_timestamp_ms, end_timestamp_ms)
    transfer_aggregates = compute_transfer_aggregates(transfers)
    timestamp_data = self.adapter.read_transfer_timestamps(  # ← FAILS HERE
        start_timestamp_ms, end_timestamp_ms, addresses
    )
```

### 4. Consumer: Burst Detection
**Location**: [`src/chainswarm_analyzers_baseline/patterns/detectors/burst_detector.py:53-60`](src/chainswarm_analyzers_baseline/patterns/detectors/burst_detector.py:53)

The `BurstDetector` uses this data structure:
```python
for node in G.nodes():
    node_timestamps = timestamp_data.get(node, [])  # List[Dict[str, Any]]
    # Each dict has: {'timestamp': int, 'volume': float, 'counterparty': str}
```

### 5. Protocol Gap
**Location**: [`src/chainswarm_analyzers_baseline/protocols/io.py:4-34`](src/chainswarm_analyzers_baseline/protocols/io.py:4)

The `InputAdapter` protocol does **NOT** define `read_transfer_timestamps()`, meaning:
- It's not enforced as a required method
- ParquetAdapter can technically comply with the protocol without it
- But the pipeline assumes all adapters have it

## Impact Analysis

### Current State
```mermaid
graph TB
    A[Pipeline.run] -->|run_patterns=True| B[read_transfers]
    B --> C[compute_transfer_aggregates]
    C --> D[read_transfer_timestamps]
    D -->|ParquetAdapter| E[❌ AttributeError]
    D -->|ClickHouseAdapter| F[✅ Success]
    
    style E fill:#f99,stroke:#f00
    style F fill:#9f9,stroke:#0f0
```

### Feature vs Pattern Execution
```mermaid
graph LR
    subgraph "Features Only"
        F1[read_money_flows] --> F2[build_graph]
        F2 --> F3[analyze_features]
        F3 --> F4[✅ Works]
    end
    
    subgraph "Patterns Only"
        P1[read_money_flows] --> P2[read_transfers]
        P2 --> P3[read_transfer_timestamps]
        P3 --> P4[❌ Fails for Parquet]
    end
    
    style F4 fill:#9f9,stroke:#0f0
    style P4 fill:#f99,stroke:#f00
```

## Solution Architecture

### Design Principles
1. **Adapter Parity**: Both adapters should support the same interface
2. **Data Reuse**: Leverage already-loaded `transfers.parquet` data
3. **Performance**: Minimize redundant I/O operations
4. **Protocol Compliance**: Update protocol to enforce consistency

### Proposed Solution: Implement Missing Method

#### Option A: Standard Implementation (RECOMMENDED)
Implement `read_transfer_timestamps()` in `ParquetAdapter` by transforming already-loaded transfer data.

**Advantages**:
- ✅ Maintains adapter parity
- ✅ Reuses cached DataFrame (`self._transfers_df`)
- ✅ No additional file I/O required
- ✅ Consistent with ClickHouse implementation

**Implementation Approach**:
```python
def read_transfer_timestamps(
    self,
    start_timestamp_ms: int,
    end_timestamp_ms: int,
    addresses: List[str]
) -> Dict[str, List[Dict[str, Any]]]:
    """
    Extract timestamp data for burst detection from transfers.
    
    Returns dict mapping address -> list of {timestamp, volume, counterparty}
    """
    # 1. Load and filter transfers (reuses cache)
    transfers = self.read_transfers(start_timestamp_ms, end_timestamp_ms)
    
    # 2. Build address -> timestamp records mapping
    address_timestamps = defaultdict(list)
    
    for transfer in transfers:
        from_addr = transfer['from_address']
        to_addr = transfer['to_address']
        timestamp = transfer['block_timestamp']
        volume = transfer.get('amount_usd', 0) or 0
        
        # Add record for from_address (if in our address list)
        if from_addr in addresses:
            address_timestamps[from_addr].append({
                'timestamp': timestamp,
                'volume': float(volume),
                'counterparty': to_addr
            })
        
        # Add record for to_address (if in our address list)
        if to_addr in addresses:
            address_timestamps[to_addr].append({
                'timestamp': timestamp,
                'volume': float(volume),
                'counterparty': from_addr
            })
    
    return dict(address_timestamps)
```

#### Option B: Protocol Update (COMPLEMENTARY)
Update `InputAdapter` protocol to mandate this method.

**Changes to [`protocols/io.py`](src/chainswarm_analyzers_baseline/protocols/io.py:4)**:
```python
class InputAdapter(Protocol):
    # ... existing methods ...
    
    def read_transfer_timestamps(
        self,
        start_timestamp_ms: int,
        end_timestamp_ms: int,
        addresses: List[str]
    ) -> Dict[str, List[Dict[str, Any]]]:
        """
        Read timestamp data for specific addresses.
        
        Returns:
            Dict mapping address to list of timestamp records.
            Each record: {'timestamp': int, 'volume': float, 'counterparty': str}
        """
        ...
```

### Data Flow After Fix

```mermaid
sequenceDiagram
    participant Pipeline
    participant ParquetAdapter
    participant DataFrame
    participant BurstDetector
    
    Pipeline->>ParquetAdapter: read_transfer_timestamps(start, end, addresses)
    ParquetAdapter->>ParquetAdapter: read_transfers() [uses cache]
    ParquetAdapter->>DataFrame: Filter & transform
    DataFrame-->>ParquetAdapter: Transfer records
    ParquetAdapter->>ParquetAdapter: Group by address
    ParquetAdapter-->>Pipeline: Dict[address -> timestamps]
    Pipeline->>BurstDetector: analyze(timestamp_data)
    BurstDetector-->>Pipeline: Burst patterns
```

### Performance Considerations

**Memory Impact**:
- Low - reuses already-loaded `self._transfers_df`
- Transformation is in-memory grouping operation

**Time Complexity**:
- O(T) where T = number of transfers in time window
- Already filtered by timestamp in `read_transfers()`

**Caching Strategy**:
```python
# Current cache structure (no changes needed)
self._transfers_df: Optional[pd.DataFrame] = None  # ← Reused
```

## Implementation Plan

### Phase 1: Core Implementation
1. Add `read_transfer_timestamps()` to `ParquetAdapter`
2. Add unit tests for the new method
3. Verify integration with burst detector

### Phase 2: Protocol Enhancement
1. Update `InputAdapter` protocol definition
2. Ensure both adapters satisfy updated protocol
3. Add type checking validation

### Phase 3: Validation
1. Run existing test suite
2. Test with sample Parquet data
3. Verify pattern detection works end-to-end

## Testing Strategy

### Unit Tests
```python
def test_parquet_read_transfer_timestamps():
    """Test that ParquetAdapter correctly extracts timestamp data."""
    adapter = ParquetAdapter(input_path, output_path)
    
    addresses = ["addr1", "addr2"]
    timestamp_data = adapter.read_transfer_timestamps(
        start_timestamp_ms=1000,
        end_timestamp_ms=2000,
        addresses=addresses
    )
    
    # Verify structure
    assert isinstance(timestamp_data, dict)
    assert all(addr in timestamp_data for addr in addresses)
    
    # Verify record format
    for addr, records in timestamp_data.items():
        for record in records:
            assert 'timestamp' in record
            assert 'volume' in record
            assert 'counterparty' in record
```

### Integration Tests
```python
def test_pattern_detection_with_parquet():
    """Test full pattern detection pipeline with ParquetAdapter."""
    adapter = ParquetAdapter(sample_data_path, output_path)
    pipeline = create_pipeline(adapter, network="test", settings)
    
    result = pipeline.run(
        start_timestamp_ms=start,
        end_timestamp_ms=end,
        window_days=30,
        processing_date="2026-01-05",
        run_patterns=True
    )
    
    assert result['patterns_count'] >= 0  # Should not crash
```

## Risk Assessment

| Risk | Probability | Impact | Mitigation |
|------|------------|--------|------------|
| Data structure mismatch | Low | High | Follow ClickHouse implementation exactly |
| Performance degradation | Low | Medium | Reuse cached DataFrames |
| Missing edge cases | Medium | Medium | Comprehensive unit tests |
| Protocol breaking changes | Low | Low | Backward compatible addition |

## Rollout Plan

### Immediate (Critical Fix)
1. ✅ Implement `read_transfer_timestamps()` in ParquetAdapter
2. ✅ Add basic tests
3. ✅ Verify pattern detection works

### Short-term (Protocol Enhancement)
1. Update `InputAdapter` protocol
2. Add comprehensive test suite
3. Update documentation

### Long-term (Architectural)
1. Consider extracting timestamp logic to shared utility
2. Evaluate caching strategies for large datasets
3. Add performance benchmarks

## Success Criteria

- ✅ ParquetAdapter implements `read_transfer_timestamps()`
- ✅ Pattern detection runs without errors
- ✅ Burst patterns are correctly detected
- ✅ All tests pass
- ✅ Performance is acceptable (< 5% overhead)
- ✅ Code follows existing patterns and style

## Appendix: Code Locations Reference

| Component | File | Lines |
|-----------|------|-------|
| ParquetAdapter | `adapters/parquet.py` | 14-274 |
| ClickHouseAdapter.read_transfer_timestamps | `adapters/clickhouse.py` | 108-151 |
| Pipeline usage | `pipeline/baseline.py` | 73-75 |
| BurstDetector usage | `patterns/detectors/burst_detector.py` | 53-60, 122-124 |
| InputAdapter protocol | `protocols/io.py` | 4-34 |
| Transfer aggregates | `aggregates/transfer_aggregates.py` | - |
