# 🚀 SandevistanLR — ECG-Inspired Cyclical Learning Rate

![Python](https://img.shields.io/badge/python-3.8%2B-success.svg)
![PyTorch](https://img.shields.io/badge/pytorch-2.x-orange.svg)
![TensorFlow](https://img.shields.io/badge/tensorflow-2.x-orange.svg)
[![License](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)
[![Contributions Welcome](https://img.shields.io/badge/contributions-welcome-brightgreen.svg)](CONTRIBUTING.md)

---

## 🧠 Overview

🚀 **Abstraction**  
SandevistanLR is a **step-level learning-rate scheduler** in which the learning rate evolves as a **deterministic, bounded time-domain signal**.

⚡ **Core Idea**  
Replace triangle or cosine schedules with a **physiologically inspired ECG waveform (P–QRS–T)** that explicitly encodes warm-up, exploration, and stabilization.

📈 **Control Surface**
- explicit phase boundaries
- bounded energy injection
- deterministic recovery

🔁 **Execution Model**
- evaluated per batch
- no optimizer state coupling
- no runtime heuristics

---

## 🫀 ECG Reference (P–QRS–T)

![Official ECG P–QRS–T components](https://upload.wikimedia.org/wikipedia/commons/thumb/9/9e/SinusRhythmLabels.svg/1280px-SinusRhythmLabels.svg.png)

Public-domain reference:  
[SinusRhythmLabels.svg](https://commons.wikimedia.org/wiki/File:SinusRhythmLabels.svg) — Wikimedia Commons

---

## 📊 Learning-Rate Signal

![SandevistanLR Preview](figures/01_default_normal_x1.png)

🧩 **Signal Semantics**

- **P wave** → controlled warm-up
- **QRS complex** → short, high-energy exploration burst
- **T wave** → stabilization and consolidation

The waveform is:
- bounded
- repeatable
- phase-aware
- deterministic

---

## ⚙️ Optimization Modes

🎛 **Explicit schedulable regimes**

- **normal** — baseline ECG rhythm
- **run** — accelerated rhythm with higher exploration energy
- **die** — flatline (very low constant learning rate)
- **aed** — flatline with controlled shock spikes

Modes are:
- defined in advance
- deterministic
- never inferred from loss

---

## 🧪 Project Status & Scope

🚧 **Status**  
This project is **actively under development**.

🎯 **Goal**  
Produce a **research-grade learning-rate scheduler** supported by **strong empirical evidence** across **vision and tabular benchmarks**, with results packaged for **conference-ready submission** (e.g., CVPR / NeurIPS / ICLR workshops).

---

## 🎯 Motivation — LLMs & Short Training Runs

⚙️ **Step-level execution**  
SandevistanLR operates **per batch**, not per epoch.

This is particularly relevant for **large-scale training and fine-tuning**, where runs are often limited to **very few epochs (sometimes only one)**.

📉 In these regimes:
- epoch-level LR schedules become too coarse
- warm-up + decay collapses to near-constant behavior
- restart-based heuristics lack fine-grained phase control

🫀 **ECG-style “heartbeat” cycles** provide:
- controlled exploration
- bounded instability
- deterministic stabilization

even under **extremely tight training budgets**.

---

## 🧭 Preset Configurations (Recommended)

SandevistanLR ships with **ready-to-use preset configurations** covering a wide range of optimization behaviors:

- warmup → normal → cooldown
- athlete-like training cycles
- intermittent flatlines
- die → aed → recovery protocols
- interval training
- stress-test / arrhythmia-style schedules

These presets encode **common optimization strategies** in a **reproducible and inspectable form**.

---

## 🗂 Where to Find the Presets

📁 Preset definitions live in:

- [src/sandevistanlr/configs.py](src/sandevistanlr/configs.py)

This file defines a dictionary of **named configurations (`CONFIGS`)** and their associated:
- schedule plans
- mode sequences
- timing parameters
- learning-rate bounds

Presets are **strong starting points**, not constraints.

SandevistanLR is **fully parameterizable**, allowing users to construct **arbitrarily complex schedules** by tuning:
- LR bounds
- cycle length
- ECG morphology
- mode transitions

to match their **training budget** and **stability requirements**.

---

## 🎞 Visual Previews

All preset waveforms are **pre-generated** and available in:

- [figures/](figures/)

Representative examples rendered below:

### 📊 Default ECG (normal)

![Default ECG (normal)](figures/01_default_normal_x1.png)

---

### 📊 Warmup → Normal → Cooldown

![Warmup Normal Cooldown](figures/02_warmup_normal_cooldown.png)

---

### 📊 Die → AED → Recovery

![Die AED Recovery](figures/05_die_aed_recovery.png)

---

### 📊 High-Energy Run Mode

![High-Energy Run Mode](figures/10_high_energy_run_mode.png)

These visualizations are intended to provide **immediate, pre-training insight** into each schedule’s behavior, enabling reasoning about optimization dynamics **before integration into training pipelines**.


---

## 📦 Installation

🚀 From PyPI

    pip install sandevistanlr

🔧 From source

    git clone https://github.com/takfarine/SandevistanLR.git

---

## 🔌 Usage

### ⚙️ PyTorch — Per-Step Control

    from sandevistanlr import SandevistanScheduleBase
    from sandevistanlr.adapters.pytorch import SandevistanLR

    base = SandevistanScheduleBase(
        steps_per_epoch=100,
        epoch_plan=[("normal", 1), ("run", 6)],
        lr_min=1e-6,
        lr_max=2e-4,
    )

    sched = SandevistanLR(base)

    for step in range(total_steps):
        lr = sched(step)
        for g in optimizer.param_groups:
            g["lr"] = lr


---

### 🔌 TensorFlow / Keras — Native Schedule

    from sandevistanlr import SandevistanScheduleBase, SandevistanLRTF

    base = SandevistanScheduleBase(
        steps_per_epoch=100,
        epoch_plan=[("normal", 1), ("run", 6)],
        lr_min=1e-6,
        lr_max=2e-4,
    )

    optimizer = tf.keras.optimizers.AdamW(
        learning_rate=SandevistanLRTF(base)
    )


---

## 📐 Mathematical Formulation

📘 Full derivation is provided in:

- [code_examples/sandevistanlr-mathematical-idea.ipynb](code_examples/sandevistanlr-mathematical-idea.ipynb)

Separation of concerns:
- README → systems intuition & usage
- Notebook → equations & proofs

---

## 🧩 Design Principles

Core guarantees:

1. deterministic schedules
2. signal-based formulation
3. explicit phase control
4. cross-framework numerical parity

Suitable for:
- fine-tuning
- short-budget training
- controlled ablation studies
- reproducible benchmarks

---

## 🛣 Status & Roadmap

Current status: **v0.1.0 — stable core**

Planned:
- benchmark reports (vision, tabular)
- additional presets
- optional integration helpers

Not planned:
- heuristic auto-tuning
- randomized schedules
- opaque optimizer logic

---

## 📚 References

### 📈 Learning-Rate Scheduling

- Leslie N. Smith, Cyclical Learning Rates for Training Neural Networks (2015)  
  https://arxiv.org/abs/1506.01186

- Leslie N. Smith & Nicholay Topin, Super-Convergence (2017)  
  https://arxiv.org/abs/1708.07120

- Ilya Loshchilov & Frank Hutter, SGDR (2016)  
  https://arxiv.org/abs/1608.03983

- fastai documentation, 1cycle policy  
  https://docs.fast.ai/callback.schedule.html

### 🫀 ECG Foundations

- Willem Einthoven, Nobel Lecture (1925)  
  https://www.nobelprize.org/prizes/medicine/1924/einthoven/lecture/

- Rivera-Ruiz et al., Einthoven’s String Galvanometer (2008)  
  https://pmc.ncbi.nlm.nih.gov/articles/PMC2435435/

- Fisch, Centennial of the Electrocardiogram (2000)  
  https://www.sciencedirect.com/science/article/pii/S0735109700009761

---

## 📜 License

Apache License 2.0  
https://www.apache.org/licenses/LICENSE-2.0
