# JKP CTF Python Implementation Checklist

## Purpose

This document is an operational checklist for implementing a **Python submission** to the JKP Common Task Framework (CTF). It consolidates the official requirements from the CTF **Rules**, **Dataset Access**, and **Submit** pages into a single working note.

It is written for actual implementation, not for vague inspiration. The point is to reduce the probability of building a clever model that later fails because of a formatting rule, a hidden lookahead violation, or a submission packaging mistake. Humans are very talented at that.

---

## 1. The core objective

The CTF expects a model that takes the provided dataset and returns **monthly portfolio weights** using only information available at or before each month `t`.

At the highest level, your submission must satisfy six conditions:

1. **No lookahead bias**.
2. **No external data**.
3. **Feature selection must be algorithmic**.
4. **Code must be self-contained and reproducible**.
5. **The `main()` function and returned schema must exactly match the required interface**.
6. **The uploaded files must match the accepted submission formats**.

---

## 2. Official inputs and what they mean

The official CTF dataset access page specifies three WRDS tables:

- `contrib_global_factor.ctff_features`
- `contrib_global_factor.ctff_chars`
- `contrib_global_factor.ctff_daily_ret`

These are the only inputs you should rely on for your CTF model.

### 2.1 Required WRDS access

Before implementation, verify that you have:

- a valid **WRDS account**
- access to:
  - **CRSP Monthly Stock File**
  - **CRSP Daily Stock File**
  - **Compustat North America**

### 2.2 Python-side packages suggested by the official data page

The dataset access page explicitly shows a Python setup using:

- `pandas`
- `sqlalchemy`
- `psycopg2-binary`
- `keyring`

A typical WRDS access workflow is:

1. store WRDS credentials securely in the OS keychain via `keyring`
2. connect to WRDS Postgres via `sqlalchemy`
3. fetch the three CTF tables into DataFrames

### 2.3 Immediate data-access checklist

Before writing any model code, confirm all of the following:

- [ ] WRDS username/password are stored and retrievable via `keyring`
- [ ] WRDS Postgres connection works from Python
- [ ] all three CTF tables can be downloaded
- [ ] `id` and date-related columns are parsed correctly
- [ ] local copies of the three tables are available for exploratory development
- [ ] row counts and basic uniqueness checks were run on the downloaded data

---

## 3. Research-rule compliance checklist

This is the most important section. A fancy model with rule violations is just a more complicated invalid submission.

### 3.1 Rule 1: Temporal integrity

All information used to construct weights at time `t` must be available **at or before `t`**.

This means you **cannot**:

- use future returns in feature construction
- use future characteristics in any way
- choose hyperparameters based on later test-period performance
- manually settle on variables because you know they did well historically
- let any target leakage sneak into preprocessing or ranking

The rules page says CTF uses automated checks by comparing outputs from the **full dataset** and a **truncated dataset**. If your model produces different weights for the same month depending on whether future data was present during training, that is evidence of lookahead bias.

#### Practical implementation rules

- Never fit on the entire panel and then slice weights by date.
- Every month’s output must come from an information set truncated at that month.
- Any rolling statistic must be computed only from historical data.
- Hyperparameter tuning must be inside the historical loop, not done once using later performance knowledge.

#### Temporal-integrity checklist

- [ ] every feature uses only information available by month `t`
- [ ] every model fit for month `t` uses only data up to `t`
- [ ] every rolling or expanding statistic is history-only
- [ ] no test-period information enters preprocessing choices
- [ ] no manual ex post variable choice is embedded in the submission
- [ ] full-data vs truncated-data consistency was tested locally

### 3.2 Rule 2: Feature selection must be algorithmic

The rules explicitly prohibit **manual feature selection based on historical performance knowledge**.

So this is dangerous:

- “I will use momentum and book-to-market because I know they work.”

This is acceptable in principle:

- “At each month `t`, I algorithmically rank or select features using only information available up to `t`.”

#### Practical interpretation

You may create transformed, filtered, ranked, interacted, or selected signals, but the selection rule must be:

- explicit in code
- reproducible
- time-consistent
- based only on information available at that moment

#### Feature-selection checklist

- [ ] feature selection rule is fully coded
- [ ] no manually curated list based on known historical winners is hardcoded
- [ ] selection is recomputed using only current historical information
- [ ] transformed features are derivable only from CTF inputs

### 3.3 Rule 3: No external data

Only the provided CTF dataset is allowed.

Not allowed:

- macroeconomic datasets
- alternative data
- web-scraped data
- external factor libraries
- private labels or proprietary scores

Allowed:

- transformations of the provided variables
- moving averages, interactions, ratios, lags, ranks, summary statistics computed from the provided data

#### External-data checklist

- [ ] no external merge is used anywhere in the pipeline
- [ ] no external file is read at runtime
- [ ] all engineered variables are generated only from `chars`, `features`, and `daily_ret`
- [ ] model documentation clearly states that no external data are used

### 3.4 Rule 4: Reproducibility

Submissions must be **fully reproducible**. The rules recommend precise dependency specification.

For Python, the accepted dependency files are:

- `requirements.txt`
- `pyproject.toml`

The rules recommend exact version pinning whenever possible.

#### Reproducibility checklist

- [ ] model runs from a clean environment
- [ ] dependencies are fully declared
- [ ] package versions are pinned where needed
- [ ] random seeds are fixed
- [ ] code path is deterministic conditional on the same inputs
- [ ] methodology PDF explains the procedure clearly enough to reproduce

### 3.5 Rule 6: Portfolio construction freedom, but monthly rebalancing required

The rules state that portfolios must be **rebalanced monthly**. Beyond that, the framework does **not** impose constraints on:

- shorting
- leverage
- position limits
- turnover constraints

Those are your design choices.

#### Portfolio-construction checklist

- [ ] the model outputs one cross-section of weights per month
- [ ] weight logic is monthly, not daily or irregular
- [ ] any leverage or shorting design is intentional and documented
- [ ] turnover control, if used, is coded with only historical information

---

## 4. Execution environment checklist

The rules page describes the execution environment. You should design to those constraints rather than assuming your own machine is the universe.

### 4.1 Pre-installed Python packages

The rules currently list these Python packages as pre-installed:

- `pandas`
- `numpy`
- `pyarrow`
- `boto3`
- `scipy`
- `scikit-learn`
- `polars`
- `joblib`

This means a reasonably strong baseline stack is already present.

#### Package strategy

Prefer the pre-installed packages unless there is a compelling reason not to. Every extra dependency increases fragility and security-review risk.

#### Dependency checklist

- [ ] baseline model works using pre-installed packages only, if possible
- [ ] extra packages are strictly necessary
- [ ] extra packages are declared in `requirements.txt` or `pyproject.toml`
- [ ] no private repositories or local wheel files are required
- [ ] added packages are available on PyPI

### 4.2 Resource limits

The rules page currently states the execution resources are:

- **32 CPU cores**
- **300 GB RAM**
- **24-hour time limit**
- **1 GPU available upon request**

These values may change, so verify them before the final submission.

#### Resource-use checklist

- [ ] full-data runtime was profiled locally or approximately estimated
- [ ] peak memory usage was considered
- [ ] large merges are minimized
- [ ] repeated recomputation inside monthly loops was reduced where possible
- [ ] numeric dtypes were checked for memory efficiency
- [ ] GPU is requested only if genuinely necessary

### 4.3 Parallel processing warning

The rules explicitly warn against using `joblib` with the `threading` backend. The recommended backend is `loky`.

#### Parallelization checklist

- [ ] if using `joblib`, backend is `loky`
- [ ] nested thread deadlock risks were considered
- [ ] serial fallback works
- [ ] parallel code produces deterministic output

### 4.4 Network isolation

The environment is network-isolated at runtime.

That means:

- no outbound internet access
- no API calls
- no runtime downloads
- no remote model pulls

Build-time package installation is possible, but runtime networking is not.

#### Network-isolation checklist

- [ ] model does not call any API or URL at runtime
- [ ] no remote weight download or Hugging Face fetch is required
- [ ] no cloud storage dependency exists at runtime
- [ ] all needed data are consumed only through the function arguments

---

## 5. Submission interface checklist

This is where many avoidable failures happen.

### 5.1 Exact `main()` function signature

For Python, the submission must define:

```python
def main(chars: pd.DataFrame, features: pd.DataFrame, daily_ret: pd.DataFrame) -> pd.DataFrame:
    ...
```

This signature must match exactly.

#### Function-interface checklist

- [ ] function name is exactly `main`
- [ ] function takes exactly three DataFrame arguments: `chars`, `features`, `daily_ret`
- [ ] function returns a DataFrame
- [ ] no wrapper script is required for the judge to call the model

### 5.2 Exact output schema

The returned DataFrame must have exactly these columns:

- `id`
- `eom`
- `w`

With the following intended meanings:

- `id`: security identifier from the input data
- `eom`: end-of-month date
- `w`: portfolio weight

The rules also state:

- output must be **non-empty**
- column names must be exactly `id`, `eom`, `w`
- no missing values are allowed
- serialized output size must not exceed **50 MB**

The rules further state that the pipeline automatically captures the function return value. You should **not** try to write output files directly.

#### Output-schema checklist

- [ ] returned DataFrame has columns exactly `id`, `eom`, `w`
- [ ] `id` values are passed through unchanged from the input universe
- [ ] `eom` is valid month-end date data
- [ ] `w` is numeric and finite
- [ ] no missing values exist in any output column
- [ ] output is not empty
- [ ] output file size estimate is below 50 MB
- [ ] code does not manually write the judged output file

### 5.3 Test-period handling

Only rows where `ctff_test == True` affect scoring. However, your model may output weights for all dates.

#### Practical recommendation

You may choose either of the following approaches:

1. return weights for **all dates**, letting the system score only the test rows, or
2. return only the relevant dates if your implementation is cleaner

In either case, make sure your logic is consistent and documented.

#### Test-period checklist

- [ ] handling of `ctff_test` is understood
- [ ] output date coverage is intentional
- [ ] no scoring-period logic introduces leakage into training

### 5.4 File constraints

The rules page specifies a maximum source file size of **1 MB per source file**, UTF-8 encoding, and source-only files.

#### File-constraint checklist

- [ ] source files are under 1 MB each
- [ ] files are UTF-8 encoded
- [ ] no binary content is embedded in the source file
- [ ] submission script is concise enough to pass validation

---

## 6. Validation-mode and full-mode checklist

The rules page says the execution system has two modes:

- **Validation mode**: small subset, about **4 MB**, with **123 monthly observations**
- **Full mode**: complete dataset, about **1.1 GB**

The crucial point is that **your real model logic must still run in validation mode**.

This is not a dummy smoke test for a toy branch. If your model requires a minimum sample length, it must still function on at least 123 months.

### 6.1 Why this matters

A model that assumes a 15-year rolling training window, or a huge warm-up requirement, may fail validation if you do not code a fallback.

### 6.2 Validation-mode checklist

- [ ] the core model runs on 123 months without crashing
- [ ] warm-up logic exists for short samples
- [ ] rolling windows degrade gracefully if the sample is short
- [ ] no branch returns an empty DataFrame in validation mode
- [ ] validation-mode output still respects the required schema
- [ ] local tests were run on reduced data to simulate validation

### 6.3 Recommended fallback patterns

If the available training window is shorter than your preferred horizon, use one of these explicit fallback rules:

- expanding window until the preferred rolling window length becomes available
- shorter rolling window with a fixed minimum threshold
- simple default allocator for early months
- rank-based or heuristic model until enough observations accumulate

The fallback rule should be coded and documented, not improvised mentally.

---

## 7. Security checklist

The rules page lists prohibited operations. If these appear in your source code, the submission may be rejected.

### 7.1 Prohibited categories

The rules explicitly prohibit patterns involving:

- network operations such as `requests`, `urllib`, `socket`, `http.client`
- shell execution such as `subprocess`, `os.system`, `os.popen`
- dynamic code execution such as `eval()`, `exec()`, `compile()`
- unsafe filesystem access outside allowed temporary locations
- hardcoded credentials or secret exposure

### 7.2 Security checklist

- [ ] no `requests` or similar runtime network client appears in the code
- [ ] no `subprocess` or shell call appears in the code
- [ ] no `eval`, `exec`, or dynamic code execution appears
- [ ] no secret, token, password, or private path is embedded
- [ ] temporary file usage, if any, is minimal and safe
- [ ] code is clean enough to pass a basic security scan

### 7.3 Dependency security

The rules also state that additional packages are scanned for vulnerabilities and malware indicators.

#### Dependency-security checklist

- [ ] avoid obscure packages unless truly necessary
- [ ] keep extra dependencies minimal
- [ ] prefer standard, maintained packages
- [ ] pin versions if using additional dependencies

---

## 8. Submit-page checklist

The submit page describes what must be uploaded.

### 8.1 Required and optional files

The submit page states the following:

- **Model Script**: required
- **Model Weights CSV**: required
- **Dependencies File**: optional
- **Documentation PDF**: optional

Supported formats listed on the submit page are:

- script: `.py` or `.r`
- weights: `.csv`
- dependencies: `.txt`, `.toml`, `.lock`
- documentation: `.pdf`

Maximum file size on the submit page is **1024MB per file**.

### 8.2 Public leaderboard notice

The submit page states that qualifying submissions may appear on a **public leaderboard**, and that your:

- name
- model name
- code file
- documentation
- performance metrics

may be visible to all visitors and downloadable.

That means your submission should be treated as a public artifact.

### 8.3 Submission-package checklist

- [ ] script file is final and upload-ready
- [ ] weights CSV matches the script-generated output
- [ ] dependency file is included only if needed
- [ ] methodology PDF is included if helpful
- [ ] no private comments, file paths, credentials, or unpublished notes remain in uploaded files
- [ ] model name is clear and professional
- [ ] GPU request is checked only if truly needed
- [ ] methodology PDF mentions GPU need if requesting GPU

---

## 9. Recommended Python project structure

Below is a practical structure for a CTF Python submission project.

```text
ctf_submission/
├── model.py                # required submission script
├── requirements.txt        # optional
├── methodology.pdf         # optional but recommended
├── local_tests/
│   ├── test_schema.py
│   ├── test_no_lookahead.py
│   ├── test_small_sample.py
│   └── test_determinism.py
├── notebooks/
│   └── exploration.ipynb   # never submit this directly
└── outputs/
    └── weights_local.csv   # local debug only
```

### 9.1 Separation principle

Keep your **submitted script** very clean.

A good discipline is:

- exploratory notebooks stay out of the submission
- helper experiments stay local
- only the final clean implementation goes into `model.py`

---

## 10. Pre-submission local test checklist

Before uploading anything, run these tests locally.

### 10.1 Schema tests

- [ ] return type is `pd.DataFrame`
- [ ] columns are exactly `id`, `eom`, `w`
- [ ] `id` is integer-like
- [ ] `eom` is date-like
- [ ] `w` is numeric
- [ ] no NaN or infinite values exist
- [ ] no duplicate `(id, eom)` rows exist

### 10.2 Temporal-integrity tests

- [ ] re-run model on a truncated dataset
- [ ] compare overlapping months to the full-data run
- [ ] confirm identical results for the common periods within floating-point tolerance

### 10.3 Small-sample tests

- [ ] run model on a short sample with about 123 monthly observations
- [ ] verify fallback logic works
- [ ] confirm non-empty valid output

### 10.4 Runtime tests

- [ ] model completes within an acceptable time budget
- [ ] peak memory does not explode on realistic sample sizes
- [ ] parallel code behaves as expected

### 10.5 Determinism tests

- [ ] repeated runs with the same inputs produce the same weights
- [ ] all random seeds are fixed
- [ ] multithreading or multiprocessing does not change results

---

## 11. A recommended implementation sequence

If starting from zero, the cleanest order is the following.

### Stage 1. Data access

- connect to WRDS
- download the three official CTF tables
- inspect their schema
- save local development copies

### Stage 2. Minimal valid model

Build the simplest possible valid submission first.

For example:

- a trivial equal-weight model
- or a simple rank-based long-short signal using only one programmatically generated rule

The goal here is not brilliance. It is to verify:

- exact interface compliance
- exact output schema compliance
- successful execution on short and full-like data

### Stage 3. Temporal-safe training loop

Once the minimal model works:

- add a monthly loop
- truncate training data at each month
- fit the model inside that loop
- generate weights only from available information

### Stage 4. Fallback logic

Before making the model sophisticated:

- add short-sample fallback rules
- test validation-mode robustness
- test determinism

### Stage 5. Optimization and scaling

Only after the above works should you:

- optimize runtime
- parallelize carefully
- add richer feature engineering
- tune hyperparameters more aggressively

### Stage 6. Submission package

Prepare:

- final `model.py`
- `weights.csv`
- optional dependency file
- optional methodology PDF

Then verify that everything is public-safe.

---

## 12. Common failure modes

These are the mistakes most worth preventing.

### 12.1 Hidden lookahead through preprocessing

Examples:

- z-scoring using the full sample
- selecting variables after inspecting full test-period performance
- imputing with future cross-sections or future panel statistics

### 12.2 Validation failure due to minimum-window assumptions

Examples:

- requiring 180 months when validation offers only 123
- returning empty output during warm-up months
- code path that silently drops all rows in small samples

### 12.3 Formatting failure

Examples:

- output columns named `permno`, `date`, `weight` instead of `id`, `eom`, `w`
- returning a Polars DataFrame when the environment expects pandas-like semantics
- writing the result to disk instead of returning it

### 12.4 Submission hygiene failure

Examples:

- uploading a notebook instead of a clean script
- leaving in private comments or machine-specific paths
- forgetting that the code may appear publicly on the leaderboard

---

## 13. Minimal final checklist

If you want the shortest actionable version, this is it.

### Data

- [ ] access WRDS
- [ ] download `ctff_features`, `ctff_chars`, `ctff_daily_ret`
- [ ] verify schemas and keys

### Rules

- [ ] no lookahead
- [ ] no external data
- [ ] feature selection algorithmic only
- [ ] monthly rebalancing

### Code

- [ ] exact `main(chars, features, daily_ret)`
- [ ] returns pandas DataFrame
- [ ] exact columns `id`, `eom`, `w`
- [ ] no NaN
- [ ] non-empty output
- [ ] output under 50 MB serialized

### Environment

- [ ] runtime has no network dependency
- [ ] no prohibited operations (`requests`, `subprocess`, `eval`, etc.)
- [ ] dependencies declared if needed
- [ ] validation-mode sample handled correctly

### Submit

- [ ] upload `.py` model script
- [ ] upload weights `.csv`
- [ ] optional `requirements.txt` or `pyproject.toml`
- [ ] optional methodology `.pdf`
- [ ] remove all private or embarrassing artifacts before upload

---

## 14. Source note

This checklist was compiled from the official JKP Common Task Framework pages:

- CTF Rules
- CTF Dataset Access
- CTF Submit a Model

Because the CTF pages may be updated, you should re-check the official pages immediately before final submission, especially for:

- compute resources
- accepted file formats
- package/environment rules
- public leaderboard disclosure terms

