# sagellm-compression

Model Compression & Acceleration Module for SageLLM (Task 3).

[![CI](https://github.com/intellistream/sagellm-compression/actions/workflows/ci.yml/badge.svg)](https://github.com/intellistream/sagellm-compression/actions/workflows/ci.yml)
[![PyPI version](https://badge.fury.io/py/isagellm-compression.svg)](https://badge.fury.io/py/isagellm-compression)
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
[![codecov](https://codecov.io/gh/intellistream/sagellm-compression/branch/main/graph/badge.svg)](https://codecov.io/gh/intellistream/sagellm-compression)

## 📌 Architecture & Responsibility

`sagellm-compression` is the core module responsible for model compression (quantization, sparsity) and inference acceleration strategies (speculative decoding, CoT optimizations).

### Dependency Graph

```mermaid
graph TD
    Protocol[isagellm-protocol] --> Backend[isagellm-backend]
    Protocol --> Compression[isagellm-compression]
    Backend --> Compression
    Compression --> Core[isagellm-core]
    Compression --> KVCache[isagellm-kv-cache]
```

- **Depends on**:
  - `isagellm-protocol`: Shared schemas and definitions.
  - `isagellm-backend`: Hardware acceleration kernels (for Quantization).
- **Used by**:
  - `isagellm-core`: Main inference engine (integrates acceleration strategies).
  - `isagellm-kv-cache`: Uses compression techniques for KV storage.

## ✨ Features

- **Chain-of-Thought (CoT) Acceleration** (Task 3.5, Implemented): CoT detection + prompt compression + chain shortening with metrics.
- **Quantization** (Task 3.1, Implemented): INT8/INT4 weight and activation quantization.
- **Sparsity** (Task 3.2, Implemented): Structured (N:M / block) and unstructured pruning support.
- **Speculative Decoding** (Task 3.3, Implemented): Draft-verify orchestration with acceptance-rate metrics.
- **Kernel Fusion** (Task 3.4, Implemented): Attention/MLP fusion interfaces with CPU stub for CI.

## 📦 Installation

```bash
pip install isagellm-compression
```

### Requirements
- Python >= 3.10
- `isagellm-protocol >= 0.4.0.0`
- `isagellm-backend >= 0.4.0.0`

## 🚀 Quick Start

### Chain-of-Thought (CoT) Templates

Currently, the CoT module provides template management for reasoning tasks.

```python
from sagellm_compression.cot import CoTTemplateManager

# Initialize the manager
# Note: Ensure you have the templates directory available
manager = CoTTemplateManager(template_dir="templates/cot")

# Load a specific strategy template (e.g., zero-shot reasoning)
try:
    template_content = manager.load_template("zero_shot")
    # Render the prompt with a question
    prompt = manager.render(template_content, question="What is the result of 25 * 14?")
    print("--- Generated Prompt ---")
    print(prompt)
except FileNotFoundError:
    print("Template not found. Please ensure 'templates/cot' exists.")
```

## 🛠️ Development

### Setup

```bash
git clone git@github.com:intellistream/sagellm-compression.git
cd sagellm-compression
./quickstart.sh

# Install in editable mode with dev dependencies
pip install -e ".[dev]"
```

### Testing & Linting

```bash
# Run all tests
pytest -v

# Run issue #8 unit-test subset
pytest -v tests/unit/test_quantization.py tests/unit/test_sparsity.py tests/unit/test_speculative_config.py

# Check code style
ruff check .
ruff format .
```

### Core Principles
- **Protocol-First**: Changes involving schemas must update `isagellm-protocol` first.
- **CPU-First**: All compression logic must reference CPU implementation by default.
- **Fail-Fast**: Missing configurations must raise explicit errors.

## 📚 Documentation

- [Protocol Specification (v0.1)](https://github.com/intellistream/sagellm-docs/blob/main/docs/specs/protocol_v0.1.md)
- [Team & Roles](docs/TEAM.md)
- [Related Papers](docs/RELATED_PAPERS_SUMMARY.md)

## 🔄 贡献指南

请遵循以下工作流程：

1. **创建 Issue** - 描述问题/需求
   ```bash
   gh issue create --title "[Bug] 描述" --label "bug,sagellm-compression"
   ```

2. **开发修复** - 在本地 `fix/#123-xxx` 分支解决
   ```bash
   git checkout -b fix/#123-xxx origin/main-dev
   # 开发、测试...
   pytest -v
   ruff format . && ruff check . --fix
   ```

3. **发起 PR** - 提交到 `main-dev` 分支
   ```bash
   gh pr create --base main-dev --title "Fix: 描述" --body "Closes #123"
   ```

4. **合并** - 审批后合并到 `main-dev`

更多详情见 [.github/copilot-instructions.md](.github/copilot-instructions.md)

## 📅 Versioning & Changelog

Current Version: **0.4.0.10**

See [CHANGELOG.md](CHANGELOG.md) for full history.

## License

Private - IntelliStream Research Project
