# dyfinance Specification

## Overview

dyfinance is a **drop-in replacement** for yfinance.
It transparently adds a DuckDB-backed OHLCV cache layer, eliminating redundant fetches and reducing network overhead and response time.

```python
# Users only need to change this one line
import dyfinance as yf
```

## Design Goals

1. **Identical I/O to yfinance** - All public APIs share the same signatures, defaults, and output structures as yfinance
2. **Transparent caching** - Users never need to be aware of the cache
3. **Safe caching** - Raw data (auto_adjust=False) is stored; post-processing is applied at read time so the cache is independent of option combinations
4. **Full yfinance fallback** - Any feature not implemented by dyfinance is transparently delegated to yfinance

## yfinance Compatibility

See [yfinance-compatibility.md](yfinance-compatibility.md).

---

## Architecture

```
dyfinance (Public API)
├── Ticker          Proxy for yfinance.Ticker; overrides history() with caching
├── Tickers         Proxy for yfinance.Tickers; uses cache via download()
├── download()      Drop-in replacement for yfinance.download()
└── __getattr__     Transparently delegates everything else to yfinance

Internal Modules
├── _ticker.py      Ticker class
├── _tickers.py     Tickers class
├── _download.py    download() function
├── _cache.py       CacheEngine (DuckDB operations)
├── _schema.py      DuckDB table definitions
├── _constants.py   Column name constants
└── _version.py     Version
```

---

## Component Specifications

### Ticker (`_ticker.py`)

Proxy class for yfinance.Ticker.

- `__init__(ticker, session=None)` holds a yfinance.Ticker internally
- `__getattr__` delegates all properties and methods except `history()` to yfinance.Ticker
- Only `history()` is intercepted with caching logic

#### history() Flow

```
history(period, interval, start, end, ...) is called
    │
    ├─ interval is not cacheable → delegate directly to yfinance.Ticker.history()
    │
    ├─ interval is cacheable (1d, 5d, 1wk, 1mo, 3mo)
    │   ├─ Resolve period/start/end to UTC timestamps
    │   ├─ Check cache via CacheEngine.get_cached_ohlcv()
    │   ├─ Fetch missing_ranges from yfinance and store via CacheEngine.store_ohlcv()
    │   ├─ Retrieve final data from cache
    │   └─ Apply _apply_post_processing()
    │       ├─ auto_adjust: adjust OHLC using Adj Close / Close ratio, drop Adj Close
    │       ├─ back_adjust: back-adjust using cumulative ratio, drop Adj Close
    │       ├─ rounding: round price columns to 2 decimal places
    │       ├─ actions=False: drop Dividends / Stock Splits / Capital Gains columns
    │       └─ keepna=False: drop rows where all data columns are 0 or NaN
    └─ Return DataFrame
```

#### Cache Storage Rules

- Always queries yfinance with `auto_adjust=False, actions=True, rounding=False, keepna=False`
- Stores raw data as-is in DuckDB
- User-specified options are applied as post-processing after reading from cache

### Tickers (`_tickers.py`)

Proxy class for yfinance.Tickers.

- Holds a `{symbol: dyfinance.Ticker}` dictionary internally
- `history()` delegates to `download()`
- `download()` calls `dyfinance.download()`
- Accepts `**kwargs` (yfinance.Tickers compatibility)
- Preserves ticker input order without deduplication (same behavior as yfinance.Tickers)

### download() (`_download.py`)

Drop-in replacement for yfinance.download().

- Cacheable intervals → fetches each ticker via `dyfinance.Ticker.history()` (cache-enabled)
- Non-cacheable intervals → delegates directly to `yfinance.download()`
- Ticker normalization: `sorted(set(...))` deduplication (same behavior as yfinance.download)
- `ignore_tz`, `group_by`, `multi_level_index` handling is identical to yfinance

### CacheEngine (`_cache.py`)

DuckDB-backed OHLCV cache engine.

- Singleton pattern (`CacheEngine.default()`)
- Default DB path: `~/.dyfinance/cache.duckdb` (overridable via `DYFINANCE_DB_PATH` env var)
- `get_cached_ohlcv(ticker, interval, start, end)` → `(cached_df, missing_ranges)`
- `store_ohlcv(ticker, interval, df)` → stores to cache and updates metadata
- `cache_meta` table tracks the cached date range for each (ticker, interval) pair

#### Cache Validation (Spot Check)

When a full cache hit occurs (no missing ranges), dyfinance validates cached data freshness:

```
Full cache hit
  │
  ├─ fetched_at within STALE_THRESHOLD (default 6h) → return cached data as-is
  │
  └─ fetched_at is stale
      ├─ Fetch last 5 rows from yfinance (lightweight)
      ├─ Compare Adj Close values with cache
      │
      ├─ Match (rtol ≤ 1e-4) → update fetched_at, return cached data
      │
      └─ Mismatch
          ├─ Uniform ratio → apply ratio correction to ALL cached Adj Close
          │                   (preserves data older than yfinance's ~2yr window)
          └─ Non-uniform    → invalidate cache, re-fetch requested range
```

- `STALE_THRESHOLD`: configurable via `DYFINANCE_STALE_HOURS` env var (default: `6`)
- Spot check only fires on full cache hits; partial hits already fetch from yfinance

#### Manual Cache Invalidation

```python
Ticker("AAPL").clear_cache()          # Clear all intervals for AAPL
Ticker("AAPL").clear_cache("1d")      # Clear only daily cache for AAPL
```

#### Cacheable Intervals

```python
CACHEABLE_INTERVALS = {"1d", "5d", "1wk", "1mo", "3mo"}
```

Intraday intervals (`1m`, `2m`, `5m`, `15m`, `30m`, `60m`, `90m`, `1h`) are not cached.

### DuckDB Schema (`_schema.py`)

| Table | Primary Key | Purpose |
|-------|-------------|---------|
| `ohlcv` | (ticker, interval, ts) | OHLCV + Adj Close + dividends/splits data |
| `cache_meta` | (ticker, interval) | Tracks cached date ranges |
| `schema_version` | - | Schema version management |

---

## Dependencies

| Package | Version | Purpose |
|---------|---------|---------|
| yfinance | >=1.2, <2 | Market data source |
| duckdb | >=1.0, <2 | Local cache database |
| pandas | >=2.0 | DataFrame operations |

Python >= 3.12

---

## Test Strategy

| Layer | Files | Marker | Description |
|-------|-------|--------|-------------|
| Layer 1: Mock | `test_ticker_history.py`, `test_download.py` | (default) | Mock yfinance and unit-test dyfinance logic |
| Layer 2: Fixture | `test_compat.py` (TestFixture*) | (default, skip if missing) | Verify output structure against recorded parquet fixtures |
| Layer 3: Live | `test_compat.py` (TestLive*) | `@pytest.mark.network` | Real-time comparison between yfinance and dyfinance output |
| Proxy | `test_proxy.py` | (default) | Verify module fallback and attribute delegation |
| Cache | `test_cache.py` | (default) | CacheEngine unit tests |
| Schema | `test_schema.py` | (default) | DuckDB schema verification |

### Running Tests

```bash
# CI (no network required)
pytest tests/ -m "not network"

# Live compatibility tests (network required)
pytest tests/ -m "network"
```
