# eapctf Academic Trustworthiness Audit

**Date:** 2026-03-23
**Mode:** Knowledge Work — formula-level verification against cited papers
**Auditors:** 8 parallel Claude subagents, one per module
**Updated:** 2026-03-23 — all C1-C8 critical issues and M1-M5, M9 minor issues resolved

---

## Executive Summary

~~The eapctf package is **partially trustworthy**. The core portfolio-sorting machinery is correct. However, **6 critical issues** across 4 modules will produce results that diverge from cited papers.~~

**All critical issues (C1-C8) and minor issues (M1-M9, M10) have been resolved.** The package is now academically trustworthy for replication studies.

| Module | OK | CRITICAL | MINOR | NOTATION |
|---|---|---|---|---|
| `sorting/sorts.py` | 8 | 0 | 2 | 1 |
| `sorting/factors.py` | 9 | 0 | 2 | 0 |
| `crosssection/fmb.py` | 3 | 2 | 3 | 0 |
| `predict/oos.py` | 3 | 2 | 0 | 0 |
| `sdf/hj.py` | 6 | 0 | 1 | 0 |
| `portfolio/constructor.py` | 5 | 1 | 0 | 0 |
| `portfolio/weighting.py` | 3 | 1 | 1 | 0 |
| `predict/models.py` | 4 | 1 | 1 | 0 (+1 MISSING) |
| **TOTAL** | **41** | **6** | **10** | **1** |

---

## Critical Issues (Must Fix)

### C1 — `crosssection/fmb.py`: Sigma_f Computed from Panel Data

**Paper:** Shanken (1992), Eq. 4.
**c = 1 + λ' Σ_f^{-1} λ** where Σ_f is the **time-series** covariance matrix of factor returns.

**Code:** `factor_data = df.groupby(date_col)[factor_cols].mean()` averages factor values across entities within each date. For panel data where the same factor value appears once per entity-period, this produces the right mean but the covariance is computed from what amounts to per-date averages across the full entity cross-section. If factors are not exactly entity-invariant (or the data structure mixes factor values and firm-level exposures), Σ_f is silently biased.

**Impact:** Wrong Shanken correction factor c → wrong t_shanken for every risk price.

**Fix:** Extract the factor series directly from a date-indexed source (one row per period) before computing `np.cov()`.

---

### C2 — `crosssection/fmb.py`: `shanken_applied` Flag Is Unreliable

**Code:** `shanken_applied = shanken_correction` (the input argument). When `factor_cols is None` but `shanken_correction=True`, the correction block is silently skipped but the flag is returned as `True`.

**Impact:** Downstream code checking `result.shanken_applied` incorrectly concludes Shanken-corrected statistics are available.

**Fix:** Set `shanken_applied = True` only inside the correction block after computing c.

---

### C3 — `predict/oos.py`: OOS R² SST Uses Wrong Benchmark

**Paper:** GKX (2020), Eq. 1. The denominator uses **r̄_{t+1} = cross-sectional mean at the test period**.

**Code:** `hist_mean_val = float(train_data[ret_col].mean())` — an expanding historical mean pooled over all prior stocks and all prior periods. This single scalar is broadcast as the benchmark for every stock at every test date.

**Impact:** SST denominator drifts from GKX whenever the cross-sectional mean is non-stationary (i.e., always). The reported R²_OOS is not the GKX statistic.

**Fix:**
```python
# Inside the test-date loop, replace:
hist_mean_val = float(train_data[ret_col].mean())
# With:
hist_mean_val = float(test_data[ret_col].mean())  # cross-sectional mean at test period
```

---

### C4 — `predict/oos.py`: Primary R² Uses Ensemble-Averaged Predictions

**Paper:** GKX (2020), Eq. 1 defines R² for **a single model's** predictions.

**Code:** `OOSResult.oos_r2` accumulates SSE using `avg_pred = mean(pred_model1, pred_model2, ...)`. This ensemble-average metric has no direct mapping to GKX Eq. 1.

**Fix:** Compute and store per-model R² using each model's own SSE and the corrected SST. Report as `oos_r2_by_model` (already exists) with the correct SST, plus optionally an ensemble-average as a clearly-labelled separate field.

---

### C5 — `portfolio/constructor.py`: BSV Characteristics Not Unit-Standardized

**Paper:** Brandt, Santa-Clara, Valkanov (2009), Eq. 1. x̃_kit has **zero mean AND unit standard deviation** cross-sectionally each period.

**Code:** `vals - vals.mean()` only. Division by `vals.std()` is absent.

**Impact:** θ_k estimates absorb each characteristic's cross-sectional dispersion. Parameters are not scale-invariant and cannot be compared across characteristics or to BSV Table II.

**Fix:** Add `/ (vals.std() + 1e-8)` after `- vals.mean()` in both `fit()` (lines ~305-309) and `weights()` (lines ~398-402).

---

### C6 — `portfolio/weighting.py`: `ManagedPortfolio` Is Not MM2017

**Paper citation:** Moreira & Muir (2017), Eq. 3. The variance-managed portfolio scales returns by the inverse of lagged conditional variance: **f^VM_t = (c / σ²_{t-1}) * f_t**. No regression step, no lambda cross-validation.

**Code implements:** (1) signal standardization, (2) OLS regression of signal on external factors to form residual, (3) rolling-window Sharpe-maximizing lambda selection. This is a signal-orthogonalization + scalar management framework — structurally closer to Baz et al. (2015) "Dissecting Investment Strategies."

**Fix options:**
- (A) Rewrite `ManagedPortfolio.fit()` to implement `f^VM = c/σ²_{t-1} * f_t` (variance scaling).
- (B) Keep current design; replace MM2017 citation with an accurate description and alternative reference.

---

### C7 (MISSING) — `predict/models.py`: No OLS Predictor

**Paper:** GKX (2020), Sec. 3 explicitly includes OLS3 (plain linear regression) as the baseline benchmark.

**Code:** The model registry contains `lasso`, `ridge`, `elastic_net`, `rf`, `gbt`, `nn`, `ipca` — but no `ols` or `LinearRegression` wrapper.

**Fix:** Add `OLSPredictor` wrapping `sklearn.linear_model.LinearRegression`.

---

### C8 — `predict/models.py`: GBRT Hyperparameters Diverge from GKX

**Paper:** GKX (2020), Sec. 3 specifies: 1000 trees, max_depth=2 (stumps), learning_rate=0.01.

**Code:** 500 trees, max_depth=4, learning_rate=0.05. Also uses LightGBM with leaf-wise tree growth; GKX use level-wise depth-limited trees.

**Impact:** GBRT results will not replicate GKX Table 4.

**Fix:** Set `n_estimators=1000, max_depth=2, learning_rate=0.01`. Consider switching to `sklearn.ensemble.GradientBoostingRegressor` (or configure LightGBM with `num_leaves=3` and `max_depth=2`).

---

## Minor Issues

| # | Module | Issue | Severity | Status |
|---|---|---|---|---|
| M1 | `sorting/sorts.py` | `searchsorted(side="left")` assigns exact-breakpoint values to lower portfolio; FF convention uses upper. | MINOR | **FIXED** — changed to `side="right"` |
| M2 | `sorting/sorts.py` | `bp2_df` computed but unused in dependent bivariate sort (dependent sort correctly uses in-group quantiles). | NOTATION | Intentional; no fix needed |
| M3 | `sorting/factors.py` | `mom_factor` did not forward `exchcd_col` to `char_factor`. | MINOR | **FIXED** — added `exchcd_col` parameter forwarded to `char_factor` |
| M4 | `sorting/factors.py` | `formation_start=12` naming implied t-12 start; window actually spans t-11 to t-2. | MINOR | **FIXED** — docstring clarifies exact window (t-(start-1) to t-end) |
| M5 | `crosssection/fmb.py` | Newey-West HAC SE used instead of textbook std(λ_t)/sqrt(T); undocumented. | MINOR | **FIXED** — inline comment documents HAC usage and equivalence to textbook at lags=0 |
| M6 | `crosssection/fmb.py` | Shanken correction applied to all regressors including chars; unjustified for non-beta regressors. | MINOR | **FIXED** (prior session) — correction only applied to first n_beta columns |
| M7 | `sdf/hj.py` | Docstring did not note gross return requirement. | MINOR | **FIXED** (prior session) — gross return note added |
| M8 | `portfolio/weighting.py` | `target` param not forwarded to CVaR/CDaR optimizers. | MINOR | **FIXED** (prior session) — `efficient_risk(target)` / `min_cvar()` branching added |
| M9 | `predict/models.py` | NN: hidden layer 2=16 (not 32); dropout=0.3 (not 0.5); L1 penalty absent. | MINOR | **FIXED** — `hidden_sizes=(32,32)`, `dropout=0.5`, `l1_lambda=1e-3` added |
| M10 | `crosssection/fmb.py` | FMB intercept not in FMBResult. | MINOR | **FIXED** (prior session) — `intercept_avg` field added |

---

## Verified Correct

The following implementations are confirmed faithful to their citations:

| Component | Citation | Notes |
|---|---|---|
| NYSE breakpoints (univariate/bivariate) | BEM (2016) Ch. 5 | exchcd==1 filter; interior quantiles correct |
| VW / EW portfolio returns | BEM (2016) Ch. 5-6 | Σ(r*w)/Σ(w) and mean(r) |
| Long-short spread | BEM (2016) Ch. 5 | port_N - port_1 |
| HML = (SH+BH)/2 - (SL+BL)/2 | FF (1993) | Exact match |
| SMB_FF3 = avg(small) - avg(big) | FF (1993) | Correct |
| RMW, CMA, SMB_FF5 | FF (2015) Eqs. 1-3 | All exact |
| Momentum 12-1 window | Jegadeesh-Titman (1993) | Via log-return cumulation |
| FMB two-pass regression structure | FMB (1973) | Both passes correct |
| FMB λ̂ = mean(λ_t) | FMB (1973) | Exact |
| Shanken SE_shanken = SE * sqrt(c) | Shanken (1992) | Correct given c |
| HJ distance formula | HJ (1997) Eq. 6 | E[RR'] weighting matrix correct |
| HJ chi² test: T*d², df=N-K | HJ (1997) | Exact |
| CRRA utility + log-limit | BSV (2009) Eq. 7 | Exact with γ=1 handling |
| IPCA fit/predict | KPS (2019) via ipca pkg | indices, mean_factor=True correct |
| char_prep rank normalization | GKX (2020) App. A.1 | 2*rank/(N+1)-1 exact |
| Random Forest (300 trees, depth 6) | GKX (2020) Sec. 3 | Exact |
| Lasso / ElasticNet formula | GKX (2020) Sec. 3 | Equivalent up to penalty scaling |
| Expanding window / no lookahead | GKX (2020) Sec. 3 | Strict < on date; no contamination |

---

## References

- Bali, T. G., Engle, R. F., & Murray, S. (2016). *Empirical asset pricing.* Wiley.
- Brandt, M. W., Santa-Clara, P., & Valkanov, R. (2009). Parametric portfolio policies. *RFS*, 22(9).
- Fama, E. F., & French, K. R. (1993). Common risk factors. *JFE*, 33(1).
- Fama, E. F., & French, K. R. (2015). A five-factor asset pricing model. *JFE*, 116(1).
- Fama, E. F., & MacBeth, J. D. (1973). Risk, return, and equilibrium. *JPE*, 81(3).
- Gu, S., Kelly, B., & Xiu, D. (2020). Empirical asset pricing via machine learning. *RFS*, 33(5).
- Hansen, L. P., & Jagannathan, R. (1991). Implications of security market data. *JPE*, 99(2).
- Hansen, L. P., & Jagannathan, R. (1997). Assessing specification errors in SDF models. *JF*, 52(2).
- Kelly, B., Pruitt, S., & Su, Y. (2019). Characteristics are covariances. *JFE*, 134(3).
- Moreira, A., & Muir, T. (2017). Volatility-managed portfolios. *JF*, 72(4).
- Shanken, J. (1992). On the estimation of beta-pricing models. *RFS*, 5(1).
