# NodeStore File Descriptor Leak Fix

## Problem

The NodeStore was experiencing a severe file descriptor leak where a single process could open 180+ connections to the database file. This caused:
- `sqlite3.OperationalError: database is locked` errors
- Process crashes when file descriptor limits were reached
- Performance degradation from excessive connection overhead

## Root Cause

The original implementation used a single shared connection (`self._conn`) with `check_same_thread=False` to allow access from multiple threads (RNS announce handlers, main app thread, cleanup threads).

**Why this caused leaks:**
1. SQLite connections aren't truly thread-safe even with `check_same_thread=False`
2. The connection was created once and reused across threads
3. Some code paths (particularly in error conditions) could create multiple connections
4. Connections were never explicitly closed during normal operation
5. The `_get_conn()` method could be called multiple times without proper lifecycle management

## Solution

Replaced the persistent connection model with a **connection-per-operation** model using context managers:

### Key Changes

1. **Removed persistent `self._conn`**: No shared connection state
2. **Added `_connection()` context manager**: Guarantees cleanup
3. **Updated all methods**: Use `with self._connection() as conn:` pattern
4. **Schema initialization**: One-time connection during `__init__`
5. **Deprecated `_get_conn()`**: Kept for backwards compatibility but logs warning

### Code Pattern

**Before (LEAK):**
```python
def get_node_by_destination(self, destination_hash: str) -> MeshDevice | None:
    conn = self._get_conn()  # Reused connection, never closed
    row = conn.execute("SELECT * FROM nodes WHERE destination_hash = ?",
                      (destination_hash,)).fetchone()
    return self._row_to_device(row) if row else None
```

**After (SAFE):**
```python
def get_node_by_destination(self, destination_hash: str) -> MeshDevice | None:
    with self._connection() as conn:  # Fresh connection, auto-closed
        row = conn.execute("SELECT * FROM nodes WHERE destination_hash = ?",
                          (destination_hash,)).fetchone()
        return self._row_to_device(row) if row else None
```

## Threading Model

### Read Operations
- Lock-free (SQLite allows concurrent reads)
- Each thread gets its own connection
- No contention, maximum throughput

### Write Operations
- Use global `_db_lock` for serialization
- Prevents write conflicts
- Connection still created per-operation

## Guardrails Against Future Leaks

### 1. Connection Lifecycle Logging

Every connection creation/closure is logged at DEBUG level:
```python
logger.debug("Creating new database connection to /path/to/nodes.db")
logger.debug("Closed database connection")
```

**How to use:** Set `RNS_LOGLEVEL=7` or enable debug logging to monitor connection patterns.

### 2. Health Check Function

The `check_connection_health()` function provides runtime monitoring:

```python
from styrene.services.node_store import check_connection_health

# Call periodically (e.g., every 60 seconds)
health = check_connection_health()
if not health["healthy"]:
    logger.error(health["warning"])
```

**Thresholds:**
- 0-2 connections: Healthy (normal operation)
- 3-9 connections: Warning (potential leak or heavy usage)
- 10+ connections: Critical (definite leak)

### 3. Diagnostic Method

The `get_connection_count()` method allows ad-hoc checks:

```python
store = get_node_store()
count = store.get_connection_count()
print(f"Open connections: {count}")
```

**Usage:**
- Returns 0 in normal operation (no persistent connections)
- Returns -1 on Windows or if lsof unavailable
- Useful for debugging and testing

### 4. Context Manager Enforcement

The `_connection()` context manager ensures connections are always closed:
```python
with self._connection() as conn:
    # Connection automatically closed even if exception occurs
    result = conn.execute("SELECT ...").fetchall()
# Connection is closed here
```

## Performance Considerations

**Q: Isn't creating a connection per operation slow?**

A: No. SQLite connection overhead is negligible (<1ms) and provides:
- **Correctness**: No leaked file descriptors
- **Simplicity**: No connection pool complexity
- **Thread safety**: Each thread has its own connection
- **Reliability**: Failures are isolated to single operations

**Q: What about the cost of repeated schema checks?**

A: Schema creation happens once during `__init__` via `_ensure_schema()`. Subsequent operations don't recreate schema.

## Verification

To verify the fix works:

```bash
# Terminal 1: Run TUI
make run

# Terminal 2: Monitor connections
watch -n 1 'lsof ~/Library/Application\ Support/styrene/nodes.db | wc -l'
```

**Expected:** Connection count stays at 0-2 during normal operation.

**Before fix:** Connection count climbed to 180+ over time.

## Migration Notes

Existing code that calls `NodeStore` methods requires no changes. The API is identical:

```python
store = get_node_store()
node = store.get_node_by_destination(dest_hash)  # Works unchanged
```

Code that manually calls `store.close()` continues to work (it's now a no-op).

## Testing

Unit tests automatically benefit from the fix:
- Each test gets fresh connections
- No cross-test contamination
- Faster test execution (no connection pooling overhead)

## Monitoring in Production

Add to your monitoring setup:

```python
import logging
from styrene.services.node_store import check_connection_health

# In your main loop or background thread
def monitor_node_store():
    health = check_connection_health()
    if not health["healthy"]:
        # Alert operators
        logging.error(f"NodeStore unhealthy: {health['warning']}")
        # Optionally: send to metrics system
        metrics.gauge("node_store.connections", health["connection_count"])
```

## Commit Message

```
fix: prevent NodeStore file descriptor leaks

Replaced persistent connection model with connection-per-operation
using context managers to guarantee cleanup. This prevents the severe
file descriptor leak where a single process could open 180+ connections.

- Use _connection() context manager in all methods
- Remove shared self._conn instance variable
- Add connection lifecycle logging (DEBUG level)
- Add check_connection_health() monitoring function
- Add get_connection_count() diagnostic method
- Document threading model and guardrails

Fixes: database is locked errors, process crashes from FD exhaustion
```
