Metadata-Version: 2.4
Name: deepordinal
Version: 0.2.1
Summary: Ordinal output layers and loss functions (Rennie & Srebro, 2005) for PyTorch and TF/Keras
Author: Nicholas Hirons
License-Expression: MIT
Project-URL: Repository, https://github.com/nhirons/deepordinal
Keywords: ordinal-regression,deep-learning,pytorch,tensorflow,keras
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy
Provides-Extra: tf
Requires-Dist: tensorflow>=2.0; extra == "tf"
Provides-Extra: torch
Requires-Dist: torch>=2.0; extra == "torch"
Dynamic: license-file

# DeepOrdinal

Ordinal output layers and loss functions ([Rennie & Srebro, 2005](https://ttic.uchicago.edu/~nati/Publications/RennieSrebroIJCAI05.pdf)) for PyTorch and TF/Keras.

DeepOrdinal provides an `OrdinalOutput` layer that converts a learned logit into ordinal class probabilities via sorted thresholds, plus loss functions designed specifically for ordinal regression.

## Installation

```bash
pip install deepordinal
```

With a specific backend:

```bash
pip install "deepordinal[tf]"     # TensorFlow/Keras
pip install "deepordinal[torch]"  # PyTorch
```

For development:

```bash
pip install -e ".[tf,torch]"
```

## Backends

DeepOrdinal supports two backends with identical APIs:

| | PyTorch | TensorFlow/Keras |
|---|---|---|
| Module | `deepordinal.torch` | `deepordinal.tf` |
| Layer | `OrdinalOutput(input_dim=D, output_dim=K)` | `OrdinalOutput(output_dim=K)` |
| Loss functions | `ordinal_loss`, `ordistic_loss` | `ordinal_loss`, `ordistic_loss` |

## OrdinalOutput Layer

The `OrdinalOutput` layer accepts any input size, projects to a single logit, and converts it into K class probabilities using K-1 learned, sorted thresholds:

```
P(y = k | x) = sigmoid(t(k+1) - logit) - sigmoid(t(k) - logit)
```

where `t(0) = -inf` and `t(K) = inf` are fixed, and interior thresholds are initialized sorted.

```python
from deepordinal.torch import OrdinalOutput  # or deepordinal.tf
layer = OrdinalOutput(input_dim=16, output_dim=4)  # TF omits input_dim
```

## Loss Functions

DeepOrdinal implements the threshold-based ordinal loss functions from Rennie & Srebro, "Loss Functions for Preference Levels" (IJCAI 2005). These operate on raw logits and thresholds rather than probability output.

### `ordinal_loss`

```python
ordinal_loss(logits, targets, thresholds, construction='all', penalty='logistic')
```

- **logits**: `(batch,)` or `(batch, 1)` — raw predictor output
- **targets**: `(batch,)` — integer labels in `[0, K)`
- **thresholds**: `(K-1,)` — sorted interior thresholds
- **construction**: `'all'` or `'immediate'`
- **penalty**: `'hinge'`, `'smooth_hinge'`, `'modified_least_squares'`, or `'logistic'`
- **Returns**: scalar mean loss over the batch

#### Constructions

- **All-threshold** (default, eq 13): penalizes violations of every threshold, weighted by direction. Bounds mean absolute error. Best performer in the paper's experiments.
- **Immediate-threshold** (eq 12): only penalizes violations of the two thresholds bounding the correct class segment.

#### Penalty functions

| Name | Formula | Reference |
|---|---|---|
| `'hinge'` | `max(0, 1-z)` | eq 5 |
| `'smooth_hinge'` | 0 if z≥1, (1-z)²/2 if 0<z<1, 0.5-z if z≤0 | eq 6 |
| `'modified_least_squares'` | 0 if z≥1, (1-z)² if z<1 | eq 7 |
| `'logistic'` | `log(1 + exp(-z))` | eq 9 |

The paper recommends **all-threshold + logistic** as the best-performing combination.

### `ordistic_loss`

Probabilistic generalization of logistic regression to K-class ordinal problems (Section 4).

```python
ordistic_loss(logits, targets, means, log_priors=None)
```

- **logits**: `(batch,)` or `(batch, 1)` — raw predictor output
- **targets**: `(batch,)` — integer labels in `[0, K)`
- **means**: `(K,)` — class means (convention: μ₁=-1, μ_K=1; interior means learned)
- **log_priors**: `(K,)` or `None` — optional log-prior terms π_i
- **Returns**: scalar mean negative log-likelihood over the batch

### Example usage

#### PyTorch

```python
import torch
from deepordinal.torch import OrdinalOutput, ordinal_loss

layer = OrdinalOutput(input_dim=16, output_dim=4)
h = torch.randn(8, 16)
targets = torch.randint(0, 4, (8,))

probs = layer(h)
loss = ordinal_loss(layer.linear(h), targets, layer.interior_thresholds)
loss.backward()
```

#### TensorFlow

```python
import tensorflow as tf
from deepordinal.tf import OrdinalOutput, ordinal_loss

layer = OrdinalOutput(output_dim=4)
h = tf.random.normal((8, 16))
targets = tf.random.uniform((8,), 0, 4, dtype=tf.int32)

with tf.GradientTape() as tape:
    probs = layer(h)
    logit = tf.matmul(h, layer.kernel) + layer.bias
    loss = ordinal_loss(logit, targets, tf.squeeze(layer.interior_thresholds))
grads = tape.gradient(loss, layer.trainable_variables)
```

### Notebooks

- [`examples/example_torch.ipynb`](examples/example_torch.ipynb) — PyTorch with `ordinal_loss` and standard training loop
- [`examples/example_tf.ipynb`](examples/example_tf.ipynb) — TensorFlow/Keras with `ordinal_loss` and `GradientTape` training loop

## Running Tests

```bash
pip install -e ".[tf,torch]"
pytest -v
```

## Reference

Rennie, J. D. M. & Srebro, N. (2005). Loss Functions for Preference Levels: Regression with Discrete Ordered Labels. *Proceedings of the IJCAI Multidisciplinary Workshop on Advances in Preference Handling*.

## Changelog

### 0.2.0

- Added `ordinal_loss` — Rennie & Srebro threshold-based ordinal loss with two constructions (all-threshold, immediate-threshold) and four penalty functions (hinge, smooth hinge, modified least squares, logistic)
- Added `ordistic_loss` — ordistic negative log-likelihood loss (Rennie & Srebro, Section 4)
- Both loss functions available in `deepordinal.torch` and `deepordinal.tf`

### 0.1.0

- Added PyTorch backend (`deepordinal.torch`) with `OrdinalOutput` module
- Modernized TensorFlow backend to `tf.keras` with self-contained `OrdinalOutput` layer and `SortedInitializer`
- Dual-backend support (TensorFlow/Keras and PyTorch) with matching APIs
- `pyproject.toml` build configuration with optional `[tf]` and `[torch]` extras

### Initial

- `OrdinalOutput` Keras layer for deep ordinal regression
- Example notebook with synthetic ordinal data
