# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

## [0.1.0] - 2026-01-29

### Added
- Initial release of AtomIngest
- Core ingestion framework with YAML-driven configuration
- Dynamic model generation using AtomSQL metaprogramming
- Comprehensive normalization pipeline with 8+ built-in steps:
  - `RemoveBOMStep` - Remove UTF-8 BOM characters
  - `SkipUntilHeaderStep` - Skip metadata until header found
  - `DropRepeatedHeaderRowsStep` - Remove duplicate headers
  - `DropTrailerLinesStep` - Remove trailer/footer lines
  - `TrimWhitespaceStep` - Clean whitespace and quotes
  - `NullTokenNormalizationStep` - Convert null tokens to None
  - `ColumnNameNormalizationStep` - Standardize column names
  - `BasicTypeCleanupStep` - Parse and clean data types
- Multi-format data source support (CSV, JSON, Excel, Parquet)
- Hook system for extensible pipeline customization
- Schema drift detection and validation
- Audit metadata injection with run tracking
- Security features including PII encryption and key management
- Comprehensive error handling and reporting
- CLI interface with `atomingest` command
- Fintech-focused examples and documentation
- Comprehensive test suite with 850+ tests

### Features
- **YAML-Driven Configuration**: Define table schemas, constraints, and pipelines declaratively
- **Dynamic Model Generation**: Uses Python metaprogramming to create AtomSQL models at runtime
- **Fintech-Grade Security**: PII encryption, immutable ledgers, crypto-shredding for GDPR
- **Robust Validation**: Regex, range checks, and custom business rules
- **High Performance**: Async batching and smart buffering
- **Reliability**: Idempotency, Dead Letter Queues, transaction management

### Documentation
- Comprehensive README with quick start guide
- Detailed API documentation
- Fintech user guide with compliance patterns
- Normalization pipeline documentation and examples
- Performance tuning guide
- Security best practices

### Examples
- Basic ingestion pipeline examples
- Fintech sample projects (payment processing, market data)
- Simple and complex normalization configurations
- Hook system usage examples
- Schema validation examples

[Unreleased]: https://github.com/prashant-fintech/atomingest/compare/v0.1.0...HEAD
[0.1.0]: https://github.com/prashant-fintech/atomingest/releases/tag/v0.1.0
