# Replication Results

This page documents the performance of two reference benchmarks evaluated with
`eapctf.ctf.compute_metrics()`, alongside their corresponding entries on the CTF
leaderboard at [jkpfactors.com](https://jkpfactors.com).

## Evaluation Methodology

`compute_metrics()` mirrors the CTF server evaluation procedure:

1. **Weight alignment** — portfolio weights (identified by stock id and formation
   month-end date `eom`) are merged with daily excess returns.
2. **Monthly aggregation** — daily excess returns within each holding month are
   summed to produce monthly portfolio returns.
3. **Volatility targeting** — monthly returns are scaled to 10% annualized volatility
   (`vol_target=0.10`). This is the CTF standard and must be applied consistently
   for local metrics to be comparable to leaderboard values.
4. **Performance statistics** — Sharpe ratio (annualized), annual return (geometric),
   annualized volatility, and maximum peak-to-trough drawdown are computed over the
   full test period of 408 monthly formation dates.

The test period spans all observations where `ctff_test == 1` in the
characteristics panel.

## Benchmark Results

The table reports local `compute_metrics()` output alongside CTF leaderboard values
(fetched 2026-03-27). All statistics are computed after applying 10% vol-targeting.
"Diff %" is `(local − CTF) / |CTF|`.

| Model | Sharpe (local) | Sharpe (CTF) | Diff % | Annual Return (local) | Annual Return (CTF) | Diff % | Max Drawdown (local) | Max Drawdown (CTF) | Diff % |
|---|---|---|---|---|---|---|---|---|---|
| 1/N equal weight | 0.551 | 0.491 | +12.2% | 5.13% | 4.91% | +4.5% | -30.43% | -33.74% | -9.8% |
| IPCA (KPS 2019) | **1.939** | **1.948** | **-0.5%** | 20.64% | 19.48% | +6.0% | -10.30% | -9.92% | +3.8% |

The IPCA Sharpe ratio replicates within 0.5% of the leaderboard value. The 1/N
discrepancy (12.2% in Sharpe) reflects a larger absolute gap of 0.060 on a low
baseline value; the annual return difference of 0.22 percentage points is small in
absolute terms. Both discrepancies are consistent with minor differences in stock
universe filtering or weight normalization conventions between the local
implementation and the CTF server.

## Reproducing the Results

Both benchmarks save pre-computed weight files to `data/ctf/` after execution. To
reproduce the metrics from saved weights:

```python
import pandas as pd
from eapctf.ctf.metrics import compute_metrics

# Load pre-computed weights
ipca_w = pd.read_parquet("data/ctf/ipca_weights.parquet")
n_w = pd.read_parquet("data/ctf/one_over_n_weights.parquet")

# Load daily returns
daily = pd.read_parquet("data/ctf/ctff_daily_ret.parquet")

# Evaluate
m_ipca = compute_metrics(ipca_w, daily, vol_target=0.10)
m_n = compute_metrics(n_w, daily, vol_target=0.10)

print(f"IPCA  Sharpe: {m_ipca.sharpe:.4f}  Return: {m_ipca.annual_return:.4%}")
print(f"1/N   Sharpe: {m_n.sharpe:.4f}  Return: {m_n.annual_return:.4%}")
```

Expected output:

```
IPCA  Sharpe: 1.9389  Return: 20.6380%
1/N   Sharpe: 0.5513  Return: 5.1276%
```

To re-run the benchmarks from scratch, execute the scripts in `reference/`:

```bash
# 1/N equal weight (fast, ~10 seconds)
uv run python reference/benchmark-one-over-n.py

# IPCA (slow, requires IPCA package and several hours on large machines)
uv run python reference/benchmark-ipca-pf.py
```

The IPCA benchmark uses n_factors=5, a 120-month rolling window, and all 402
test-period features across 408 formation dates. It is parallelized via joblib and
requires the `ipca` package (`pip install ipca`).

## What the 0.009 Sharpe Difference Means

A difference of 0.009 in annualized Sharpe ratio, applied over a test period of
408 months, is within ordinary estimation noise. The CTF leaderboard Sharpe is
computed by the server on the same underlying daily returns, but the server may
apply slightly different window boundary conventions (for example, how the first
usable training window is defined) or floating-point aggregation order. The
practical implication is that local evaluation with `compute_metrics(vol_target=0.10)`
provides a reliable pre-submission estimate of CTF performance.
