# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.3.0] - 2024-10-22

### 🚀 Major Features Added

#### New Automatic Contig Extension (`teloclip extend`)
- **Intelligent genome completion**: Automatically extend draft genome contigs using telomeric overhang sequences from soft-clipped alignments
- **Quality control system**: Built-in outlier detection to prevent extension of circular genomes (mitochondria, chloroplasts)
- **Dry-run capability**: Preview proposed extensions before applying changes with `--dry-run` flag
- **Comprehensive validation**: Multi-level validation including homopolymer run detection and anchor quality assessment
- **Flexible exclusion**: Skip specific contigs by name (`--exclude-contig`) or from file lists (`--exclude-contig-file`)

#### Complete CLI Architecture Redesign
- **Modular sub-commands**: Replaced single-purpose CLI with three specialized commands:
  - `teloclip filter` - Enhanced alignment filtering with rich logging
  - `teloclip extract` - Advanced sequence extraction with multiple output formats
  - `teloclip extend` - **NEW** automatic contig extension functionality
- **Unified interface**: Consistent argument patterns and help documentation across all commands
- **Backward compatibility**: Maintains core functionality while modernizing interface

#### Memory-Efficient Streaming Architecture
- **Large genome support**: Process multi-gigabase genomes without loading entire files into memory
- **Indexed file access**: Efficient BAM/FASTA processing using pysam for contig-by-contig operations
- **Buffered I/O operations**: Optimized file writing with configurable buffer sizes

### ✨ Enhanced Features

#### Advanced Motif Analysis
- **Fuzzy matching support**: Configurable tolerance for motif variations (±1 character in homopolymer runs)
- **Comprehensive pattern counting**: Detailed motif occurrence statistics in overhang regions
- **Optimized regex processing**: Pre-compiled patterns for improved performance
- **Automatic reverse complement**: Seamless handling of both forward and reverse motif orientations

#### Rich Statistical Analysis and Reporting
- **Overhang statistics**: Detailed analysis with Z-score based outlier detection
- **Extension reports**: Human-readable summaries of proposed and applied changes
- **Terminal motif screening**: Analyze existing contig ends for telomeric sequences
- **Quality metrics**: Comprehensive validation scores for extension confidence

#### Modern Logging System
- **Rich console output**: Beautiful, informative terminal displays using the Rich library
- **Flexible log levels**: Configurable verbosity (DEBUG, INFO, WARNING, ERROR, CRITICAL)
- **File logging support**: Optional detailed log file output with timestamps
- **Context-aware messaging**: Function-specific logging with proper error context

#### Enhanced File I/O
- **Multiple output formats**: Support for both FASTA and FASTQ output with quality scores
- **Flexible organization**: Multi-file output with automatic directory creation
- **Statistics integration**: Optional overhang statistics in FASTA headers
- **Cross-platform paths**: Robust path handling using pathlib

### 🔧 Infrastructure Improvements

#### Comprehensive Test Suite
- **65 new unit tests**: Complete coverage across 4 new test modules
- **Edge case handling**: Robust testing of error conditions and malformed inputs
- **Mock-based isolation**: Proper test isolation using unittest.mock for external dependencies
- **Integration testing**: End-to-end workflow validation

#### Modern Code Architecture
- **Full type annotations**: Complete type hints throughout codebase for better maintainability
- **Dataclass integration**: Modern Python patterns for structured data handling
- **Exception handling**: Comprehensive error handling with user-friendly messages
- **NumPy-style documentation**: Extensive docstrings following established conventions

#### Updated Development Environment
- **Modern dependency management**: Updated to latest stable versions of core dependencies
- **Pre-commit integration**: Automated code quality checks with enhanced hook configurations
- **Ruff formatting**: Modern Python formatter and linter replacing Black
- **Enhanced CI/CD**: Updated GitHub Actions supporting Python 3.8-3.13

### 🔄 Breaking Changes

#### CLI Interface Changes
- **Command structure**: Changed from `teloclip [options] file` to `teloclip {filter|extract|extend} [options] files`
- **Extract behavior**: Now creates separate files per contig end by default for better organization
- **Output formatting**: New Rich-based logging provides different (improved) output formatting

#### Migration Required
```bash
# Old filter usage
teloclip --ref-idx ref.fa.fai input.sam > output.sam

# New filter usage
teloclip filter --ref-idx ref.fa.fai input.sam > output.sam

# Old extract usage
teloclip-extract --ref-idx ref.fa.fai --extract-reads input.sam

# New extract usage
teloclip extract --ref-idx ref.fa.fai input.sam
```

### 📦 Dependencies

#### Added
- **biopython**: Sequence manipulation and file format support
- **click**: Modern CLI framework with enhanced help and validation
- **pyfaidx**: Efficient FASTA file indexing and random access
- **pysam**: BAM/SAM file processing with C-speed performance
- **rich**: Beautiful terminal output and logging enhancements

#### Development Dependencies Added
- **ruff**: Modern Python linting and formatting
- **pre-commit**: Automated code quality enforcement
- Enhanced GitHub Actions workflows

### 🐛 Bug Fixes
- **SAM processing**: Fixed edge cases in soft-clip extraction and alignment validation
- **Memory management**: Resolved memory leaks in large file processing
- **Path handling**: Improved cross-platform file path management
- **Error reporting**: Enhanced error messages with actionable guidance

### 📚 Documentation
- **Comprehensive README**: Complete rewrite with usage examples for all sub-commands
- **Installation guide**: Updated instructions for new dependency requirements
- **Contributing guidelines**: Enhanced development setup with modern tooling
- **API documentation**: Complete docstring coverage for all public functions

### 🚀 Performance
- **Streaming processing**: Constant memory usage regardless of genome size
- **Compiled regex patterns**: Faster motif matching through pre-compilation
- **Indexed file access**: Dramatic speed improvements for large file processing
- **Efficient data structures**: Optimized memory layout using dataclasses and type hints

### 🎯 Quality Assurance
- **100% test pass rate**: All 65 new unit tests passing across Python 3.8-3.13
- **Cross-platform testing**: Verified compatibility on Linux, macOS, and Windows
- **Memory profiling**: Validated constant memory usage on multi-GB test files
- **Integration validation**: End-to-end testing of complete workflows

---

## Migration Guide for v0.3.0

### For Existing Filter Users (Most Common)
The core filtering functionality remains identical, just add the `filter` sub-command:

```bash
# Before v0.3.0
teloclip --ref-idx genome.fa.fai alignments.bam > filtered.sam

# v0.3.0 and later
teloclip filter --ref-idx genome.fa.fai alignments.bam > filtered.sam
```

### For Existing Extract Users
The extraction functionality is enhanced but requires the new sub-command:

```bash
# Before v0.3.0
teloclip-extract --ref-idx genome.fa.fai --extract-dir output/ alignments.bam

# v0.3.0 and later
teloclip extract --ref-idx genome.fa.fai --extract-dir output/ alignments.bam
```

### New Extension Capability
The major new feature for automatic genome completion:

```bash
# New in v0.3.0 - automatic contig extension
teloclip extend overhangs.bam genome.fa --output-fasta extended.fa --stats-report report.txt

# Preview changes without applying
teloclip extend overhangs.bam genome.fa --dry-run --stats-report preview.txt
```

---

## [Previous Releases]

### [0.2.x] - Previous versions
- Basic alignment filtering functionality
- Simple sequence extraction
- Command-line interface for telomere analysis

### [0.1.x] - Initial releases
- Core soft-clip detection algorithms
- Basic motif matching capabilities
- Foundation SAM/BAM processing tools

---

**Note**: This major release (v0.3.0) represents the largest enhancement to Teloclip since its creation, transforming it from a specialized filtering tool into a comprehensive platform for telomere analysis and genome completion. While breaking changes are introduced in the CLI interface, the core functionality remains fully compatible with simple command updates.
