# Delta

[![PyPI version](https://badge.fury.io/py/delta-lang.svg)](https://pypi.org/project/delta-lang/)
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

**A Differentiable, Constraint-Oriented Programming Language**

[**Website**](https://deltalanguage.org) | [**Documentation**](https://deltalanguage.org/docs) | [**Get Started**](https://deltalanguage.org/docs/quick-start)

Delta is a compiled language that transforms *learning intent* into optimized tensor programs. Instead of manually writing loss functions and training loops, you declare what you want your model to learn through constraints—Delta compiles this into efficient PyTorch code.

```delta
param weights = randn(784, 10);
obs images: Tensor[Float];
obs labels: Tensor[Int];

fn predict(x: Tensor[Float]) -> Tensor[Float] {
    softmax(x @ weights)
}

let pred = predict(images);
require argmax(pred) == labels;
```

This compiles to an optimized PyTorch FX graph with automatic differentiation, constraint-based loss computation, and mode-specific specialization.

---

## Key Features

### Constraint-Based Learning
Define *what* your model should learn, not *how*. Delta compiles constraints into differentiable objectives automatically.

```delta
// Hard constraint: predictions must match labels
require pred == target;

// Soft constraint: prefer smaller weights (regularization)
prefer norm(weights) < 1.0 weight 0.01;
```

### Role Annotations
Explicitly mark learnable parameters vs. observed data. Delta tracks gradient flow and validates differentiability at compile time.

```delta
param theta = randn(10, 5);   // Learnable - gradients flow here
obs x: Tensor[Float];          // Fixed input - no gradients
const scale = 0.1;             // Compile-time constant
```

### Mode Specialization
Write once, get optimized code for training, inference, and analysis:

```delta
train {
    // Only runs during training
    let dropout_mask = rand() > 0.5;
}

infer {
    // Only runs during inference - dropout disabled
    let dropout_mask = ones();
}
```

### Differentiable Control Flow
Conditionals compile to soft, differentiable operations by default:

```delta
// Compiles to: sigmoid((x - 0) / temperature) * a + (1 - sigmoid(...)) * b
if x > 0 temperature 0.1 { a } else { b }
```

---

## Installation

```bash
pip install delta-lang
```

### Requirements
- Python 3.10+
- PyTorch 2.0+ (installed automatically)

### From Source

```bash
git clone https://github.com/deltalanguage/delta.git
cd delta
pip install -e .
```

---

## Quick Start

### 1. Write a Delta Program

Create `model.delta`:

```delta
param w = randn(10, 5);
param b = zeros(5);
obs x: Tensor[Float];
obs y: Tensor[Float];

fn forward(input: Tensor[Float]) -> Tensor[Float] {
    relu(input @ w + b)
}

let pred = forward(x);
require sum((pred - y) ** 2) == 0;
```

### 2. Compile and Train

```python
import delta
import torch

# Compile the Delta program
model = delta.compile("model.delta")

# Prepare data
x_train = torch.randn(100, 10)
y_train = torch.randn(100, 5)

# Train
model.fit(
    {"x": x_train, "y": y_train},
    epochs=100,
    lr=0.01
)

# Inference
model.eval()
x_test = torch.randn(10, 10)  # Test data with 10 samples
predictions = model(x_test)
print(predictions)
```

---

## Examples

### MNIST Classification

```delta
// model.delta
param W1: Tensor = randn(784, 256) * 0.01;
param b1: Tensor = zeros(256);
param W2: Tensor = randn(256, 128) * 0.01;
param b2: Tensor = zeros(128);
param W3: Tensor = randn(128, 10) * 0.01;
param b3: Tensor = zeros(10);

fn forward(x: Tensor) -> Tensor {
    matmul(relu(matmul(relu(matmul(x, W1) + b1), W2) + b2), W3) + b3
}
```

```python
import delta
import torch
from torchvision import datasets, transforms
from torch.utils.data import DataLoader

# Flatten images to 784-dim vectors
transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Lambda(lambda x: x.view(-1))
])

train_data = datasets.MNIST('./data', train=True, download=True, transform=transform)
train_loader = DataLoader(train_data, batch_size=64, shuffle=True)

model = delta.compile("model.delta")
model.fit(train_loader, epochs=10, lr=0.001)
```

### Character-Level Transformer

See `example_projects/transformer/` for a complete implementation of a GPT-style language model in Delta.

---

## Language Reference

### Types

| Type | Description |
|------|-------------|
| `Int` | Integer |
| `Float` | Floating-point |
| `Bool` | Boolean |
| `Tensor[T]` | Tensor with element type T |
| `Tensor[Float, N, M]` | Tensor with shape |

### Declarations

```delta
param name = initializer;     // Learnable parameter
obs name: Type;               // Observed input
const name = value;           // Compile-time constant
let name = expr;              // Local binding
```

### Constraints

```delta
require expr == target;                    // Hard equality
require expr > 0;                          // Hard inequality
prefer expr < 1.0 weight 0.5;              // Soft constraint
prefer expr == target weight 1.0 slack s;  // With slack variable
```

### Control Flow

```delta
if condition { then_expr } else { else_expr }
if condition temperature 0.1 { ... }  // Soft/differentiable
for i in range(n) { ... }
while condition { ... }  // Only in non_diff blocks
```

### Mode Blocks

```delta
train { ... }     // Training-only code
infer { ... }     // Inference-only code
analyze { ... }   // Analysis-only code
non_diff { ... }  // Non-differentiable (hard control flow allowed)
```

---

## How It Works

Delta is a **staged compiler** that:

1. **Parses** Delta source into an AST
2. **Infers types** and validates roles/effects
3. **Lowers to SIR** (Semantic IR) — a differentiable-aware representation
4. **Relaxes** hard operations into soft, differentiable alternatives
5. **Compiles constraints** into penalty terms
6. **Specializes** for the execution mode (train/infer)
7. **Lowers to PyTorch FX** for execution

Delta never interprets tensor operations—it compiles everything to optimized PyTorch graphs.

```
Delta Source → AST → Typed AST → SIR → Optimized SIR → PyTorch FX → Execution
```

---

## Architecture

```
delta/
├── frontend/          # Lexer, parser, AST
├── types/             # Type system and inference
├── ir/                # Semantic IR (SIR) definition
├── passes/            # Compiler passes (relaxation, optimization)
├── backend/           # PyTorch FX lowering
├── runtime/           # Execution context and model wrapper
├── stdlib/            # Standard library (nn, optim, tensor ops)
└── compiler.py        # Main compiler orchestration
```

---

## Standard Library

### `tensor`
Core tensor operations: `zeros`, `ones`, `randn`, `concat`, `reshape`, `transpose`, etc.

### `nn`
Neural network layers: `linear`, `conv2d`, `relu`, `softmax`, `layer_norm`, `embedding`, etc.

### `optim`
Optimizers: `sgd`, `adam`, `adamw`, `learning_rate_scheduler`

### `constraints`
Constraint utilities: `equality_penalty`, `inequality_penalty`, `bound_penalty`

### `dist`
Probability distributions: `normal`, `bernoulli`, `categorical`

### `data`
Data loading utilities: `DataLoader`, `Dataset`

---

## Development

### Running Tests

```bash
# Run all tests
pytest tests/

# Run specific test file
pytest tests/test_parser.py -v

# Run with coverage
pytest tests/ --cov=delta
```

### Project Status

- [x] Lexer and parser
- [x] Type inference
- [x] SIR (Semantic IR)
- [x] PyTorch FX backend
- [x] Basic constraint compilation
- [x] Mode specialization (train/infer)
- [x] Standard library
- [ ] Probabilistic inference
- [ ] Advanced debugging (`why` command)

---

## Contributing

Contributions are welcome!

1. Fork the repository at [github.com/deltalanguage/delta](https://github.com/deltalanguage/delta)
2. Create a feature branch: `git checkout -b feature/my-feature`
3. Make your changes
4. Run tests: `pytest tests/`
5. Submit a pull request

---

## License

MIT License. See [LICENSE](LICENSE) for details.

---

## Acknowledgments

Delta builds on ideas from:
- Differentiable programming (JAX, PyTorch)
- Probabilistic programming (Stan, Pyro)
- Constraint-based learning research
- Compiler design (MLIR, XLA)
