# Akademy 

Akademy is a module containing composable object classes for developing 
reinforcement learning algorithms focused on quantitative trading and 
time-series forecasting. This module is a work-in-progress and should, at no
time, be assumed to be designed well or be free of bugs.

# Overview
Akademy is designed using an `Agent`-`Environment` model such that `Agent`-class
objects ingest information from `Environment`-class objects (`Env`), produce
an `Action`, which is then applied to the `Environment` which results in a
change in `State` and possible reward to offer feedback to the agent.

*Note*: this module does not provide any training routines -- only the object class
that can be used to support the implementation of custom training routines.

# Getting Started

To install `akademy` use the following command in the desired Python 3.7+
environment:

`pip install akademy`

Once installed, developers will have access to `Agent`, `TradeEnv`, and `Network`
class objects in which to design Reinforcement Learning algorithms to train models.

Sample training routine:

```python
from akademy.models.envs import TradeEnv
from akademy.models.agents import DQNAgent
from akademy.common.utils import load_spy_daily

# loads the dataset used during training
data = load_spy_daily(count=2500)

# load the Trading Environment
env = TradeEnv(
    data=data,
    window=50,
    asset="spy",
)

# load the agent to train
agent = DQNAgent(
    action_count=env.action_space.n,
    state_shape=env.observation_space.shape
)

# load user-defined training routine
training_routine(
    agent=agent,
    env=env
)
```

## Tests
Unit testing can be run via the following command:

`python -m unittest`

For detailed information the `--verbose` flag can be used. For more detailed 
usage consult the `unittest` module documentation.

## Available Data
This module comes with minimal data for Agents and Environments to train on.
The current data available is listed below, along with sources for the most
up-to-date versions as well:

### 1. S&P500 
Location: `/data/SPY.CSV`\
Start:  `1993-01-29`\
End:    `2023-01-23`\
Total Rows: `7,454` (excludes header)\
Header: `Date,Open,High,Low,Close,Adj Close,Volume`\
Source: https://finance.yahoo.com/quote/SPY/history?p=SPY

*note*: Any data can be used easily enough via conversion into a Pandas DataFrame
object, but must contain information for `date` and pricing data for
`open`, `high`, `low`, and `close` as well as `volume` such that each row has
at least those 6 features or the latter 5 and an index representative of date.

# Notes

## Gym vs. Gymnasium
The `Gym` project by OpenAI has been sunset and now maintained as `Gymnasium` 
by the [Farama-Foundation](https://github.com/Farama-Foundation/Gymnasium). The
`Env` classes present here make use of the newer `Gymnasium` package which, among
other differences, produces an extra item in the `step` method indicating whether
an environment has been truncated. [See here](https://github.com/Farama-Foundation/Gymnasium/blob/main/gymnasium/core.py#L63)

## PyTorch
PyTorch requires some additional consideration for setup depending on use-case.
Akademy uses an approach whereby CPU-based training and inferences are possible
via parameterized function calls. However, GPU use (e.g. CUDA) requires local
considerations. [See here] (https://pytorch.org/get-started/locally/) for a more
in-depth discussion and guide.

This module currently uses the 1.* version, though a 2.* version release
is imminent and an upgrade to that version is planned.
