# OTB-267: Fix Missing Transactions for Tastytrade Orders

**Date:** 2025-10-30  
**Version:** 0.18.0a15  
**Priority:** CRITICAL - Data Integrity Issue

## Problem

Tastytrade trades were being created and settled correctly, but **NO transactions were saved to the database**. This caused:

1. **Incorrect `realizedPNL` calculations** - Transactions are needed to calculate P&L
2. **Missing audit trail** - No record of individual leg executions
3. **Empty transactions table** for all Tastytrade orders

### Root Cause

The Tastytrade connector only emitted `orderStatusEvent` when orders were filled, but **never emitted `orderExecutionDetailsEvent`**. 

In contrast, IBKR connector emits both events:
- `orderStatusEvent` - Order status changes (FILLED, CANCELLED, etc.)
- `orderExecutionDetailsEvent` - Execution details with fill prices → **Creates transactions**

Without the execution details event, `TradeManager._onOrderExecutionDetailsEvent()` was never called, so no transactions were created in the database.

### Evidence from Logs

```
2025-10-29 09:32:01.086 | Created new trade: ID: 165 Account: 5WW18591
2025-10-29 09:32:01.996 | Order 416959878 placed successfully
2025-10-29 09:32:03.090 | Entry Order of trade 165 has been filled at $0.65 (Qty: 4)
2025-10-29 09:32:03.090 | Update on order 416959878 status OrderStatus.FILLED
2025-10-29 09:32:03.090 | (11 more FILLED status updates...)
```

→ Order filled successfully, but **no execution details emitted** → **no transactions created**

## Solution

### 1. Added Execution Details Emission

Modified `tastytradeconnector.py._update_accounts()` to emit execution details when orders are filled:

```python
if order.status == OrderStatus.FILLED and order.price is not None and order.id not in self._processed_fills:
    self._processed_fills.add(order.id)
    self._emit_execution_details_for_order(relevantOrder, order)

self._emitOrderStatusEvent(relevantOrder, order_status, filledAmount)
```

### 2. Implemented `_emit_execution_details_for_order()`

New method that extracts actual fill data from Tastytrade's `PlacedOrder.legs[].fills[]` and emits execution details events:

```python
def _emit_execution_details_for_order(self, generic_order: GenericOrder, placed_order: PlacedOrder):
    # Process each leg and its fills
    for i, broker_leg in enumerate(placed_order.legs):
        generic_leg = generic_order.legs[i]
        
        if broker_leg.fills and len(broker_leg.fills) > 0:
            # Use actual fill data from Tastytrade API
            total_fill_quantity = sum(float(fill.quantity) for fill in broker_leg.fills)
            total_fill_value = sum(float(fill.quantity) * float(fill.fill_price) for fill in broker_leg.fills)
            fill_price = abs(total_fill_value / total_fill_quantity)  # Weighted average
            fill_timestamp = broker_leg.fills[0].filled_at  # Actual execution time
        else:
            # Fallback: Use order price / number of legs (shouldn't happen)
            fill_price = abs(float(placed_order.price)) / len(placed_order.legs)
            fill_timestamp = dt.datetime.now()
        
        execution = Execution(
            id=broker_leg.fills[0].fill_id,
            action=action,
            sec_type=generic_leg.right.value,
            strike=generic_leg.strike,
            amount=int(total_fill_quantity),
            price=fill_price,  # ACTUAL fill price from exchange
            expiration=generic_leg.expiration,
            timestamp=fill_timestamp  # ACTUAL execution timestamp
        )
        self._emitOrderExecutionDetailsEvent(generic_order, execution)
```

### 3. Duplicate Prevention

Added `self._processed_fills` set to track which orders have been processed:

```python
def __init__(self):
    # ...
    self._processed_fills: set = set()  # Track processed fills to avoid duplicate transactions
```

This prevents duplicate transactions when Tastytrade sends multiple FILLED status updates for the same order.

## Tastytrade API Limitations

### Multiple FILLED Events

Tastytrade sends **multiple FILLED status updates** for the same order:
- Example: Order 416959878 received **11 FILLED events** (09:32:03 - 09:32:05)
- Each event has the same `order.id`
- Solution: Track processed order IDs in `_processed_fills` set

### Fill Data from Tastytrade API

**Good News:** Tastytrade API actually provides **individual fill details per leg**!

Each `PlacedOrder.legs[i]` contains a `fills` array with `FillInfo` objects:
- `fill_id`: Unique fill identifier
- `quantity`: Filled quantity for this specific fill
- `fill_price`: **Actual fill price for this leg** 🎯
- `filled_at`: Exact execution timestamp
- `destination_venue`: Exchange where filled

**Solution:** Use `leg.fills` to get actual leg prices instead of approximating!

```python
for broker_leg in placed_order.legs:
    for fill in broker_leg.fills:
        fill_price = fill.fill_price  # Actual price, not approximation!
        fill_quantity = fill.quantity
        fill_timestamp = fill.filled_at
```

**Benefits:**
- **Accurate leg prices** from exchange data
- **Exact execution timestamps** for audit compliance
- **No approximations needed** - actual market data
- **Handles partial fills** correctly (multiple fills per leg)

### 3. Commission and Fee Handling

**Challenge:** Tastytrade doesn't provide fee data in the order fill event stream.

**Solution:** Query transaction history API after order fill with a delay:

```python
async def _emit_commission_report_delayed(self, generic_order, placed_order):
    await asyncio.sleep(3)  # Wait for fees to post
    transactions = account.get_history(session, start_date=today - 1 day)
    # Match transactions to order legs by symbol and time
    # Emit commission report events with actual fees
```

**Fee Structure (Tastytrade):**
- Commission: $0.00 (free for options!)
- Regulatory fees: ~$0.04 per contract
- Clearing fees: ~$0.01 per contract
- **Total: ~$0.05 per contract**

## Changes

### Modified Files

1. **`optrabot/broker/tastytradeconnector.py`**
   - Added `_processed_fills` set in `__init__()`
   - Modified `_update_accounts()` to emit execution details for FILLED orders
   - Added `_emit_execution_details_for_order()` method
   - Added `_emit_commission_report_delayed()` method for async fee retrieval

## Testing

### Test Case 1: Single Leg Order
```
Order: SPX Call 5700 x 2 @ $5.50
Expected: 1 transaction with qty=2, price=$5.50
```

### Test Case 2: Multi-Leg Spread (Iron Condor)
```
Order: Iron Condor (4 legs) x 2 @ $0.65 spread price
Expected: 4 transactions, each qty=2, price=$0.1625
```

### Test Case 3: Multiple FILLED Events
```
Order: Same order ID receives 11 FILLED status updates
Expected: Only 1 set of transactions created (not 11 duplicates)
```

## Migration

**No database migration required** - This fix creates transactions that were previously missing. Existing data is not affected.

For trades that were executed before this fix (with missing transactions):
- Trade status and P&L remain correct (calculated at trade level)
- Transactions table will be incomplete for historical trades
- Consider running a recovery script if complete transaction history is needed

## Impact

### Before Fix
- ❌ Tastytrade orders: NO transactions created
- ❌ Incorrect realizedPNL (missing transaction data)
- ❌ Incomplete audit trail
- ✅ Trade status still tracked correctly

### After Fix
- ✅ Tastytrade orders: Transactions created for all legs
- ✅ Correct realizedPNL calculations
- ✅ Complete audit trail with per-leg execution records
- ✅ Duplicate prevention for multiple FILLED events

## Related Issues

- **OTB-253:** Per-broker trade recovery (v0.18.0a14)
- **OTB-236:** Shutdown RecursionError fix (v0.18.0a14)

## Notes

- This is a **critical data integrity fix**
- Affects all Tastytrade orders (past and future)
- Does NOT affect IBKR orders (already working correctly)
- Price distribution is an approximation but sufficient for audit purposes
- Actual P&L accuracy is not affected (calculated at trade level)
