# AtomIngest Documentation

Welcome to the complete documentation for **AtomIngest** - a metadata-driven data ingestion framework for fintech applications.

## 📚 Documentation Structure

### Getting Started
- [Installation Guide](installation.md) - Set up AtomIngest in your environment
- [Quick Start](quickstart.md) - Your first ingestion pipeline in 5 minutes
- [Core Concepts](concepts.md) - Understanding the framework architecture

### User Guides
- [Comprehensive User Guide](comprehensive_user_guide.md) - Complete guide from basics to production
- [Fintech User Guide](fintech_user_guide.md) - Specialized guide for financial services
- [Pipeline Configuration](pipeline_configuration.md) - Complete YAML configuration reference
- [Schema Definition](schema_definition.md) - Defining data models and field types
- [Normalization Steps](normalization.md) - Data cleaning and transformation
- [Validation](validation.md) - Data quality and business rules
- [Hooks & Custom Logic](hooks.md) - Extending pipelines with custom code

### Advanced Topics
- [Security Features](security.md) - PII encryption, audit trails, and compliance
- [Performance Tuning](performance.md) - Batch processing and optimization
- [Error Handling](error_handling.md) - DLQ, retry logic, and debugging
- [CLI Reference](cli_reference.md) - Command-line interface guide

### Examples & Recipes
- [Common Patterns](patterns.md) - Frequently used pipeline patterns
- [Fintech Examples](../examples/fintech_samples/) - Complete sample projects for financial services
  - [Payment Processing](../examples/fintech_samples/payment_processing/) - Real-time payment transaction processing
  - [Market Data Ingestion](../examples/fintech_samples/market_data/) - High-frequency market data processing
  - [Customer KYC](../examples/fintech_samples/customer_kyc/) - KYC/AML compliance workflows
  - [Risk Reporting](../examples/fintech_samples/risk_reporting/) - Regulatory reporting and risk calculations
- [Migration Guide](migration.md) - Migrating from other ETL tools

### API Reference
- [Python API](api_reference.md) - Programmatic usage
- [Field Types](field_types.md) - Complete field type reference

## 🚀 Quick Navigation

**New to AtomIngest?** Start with:
1. [Installation Guide](installation.md)
2. [Quick Start](quickstart.md)
3. [Comprehensive User Guide](comprehensive_user_guide.md)

**Building a Pipeline?** Check:
1. [Comprehensive User Guide](comprehensive_user_guide.md)
2. [Pipeline Configuration](pipeline_configuration.md)
3. [Common Patterns](patterns.md)

**Fintech Applications?** See:
1. [Fintech User Guide](fintech_user_guide.md)
2. [Fintech Examples](../examples/fintech_samples/)
3. [Payment Processing Sample](../examples/fintech_samples/payment_processing/)

**Need Help?** Look at:
1. [Troubleshooting](troubleshooting.md)
2. [FAQ](faq.md)
3. [Comprehensive User Guide](comprehensive_user_guide.md)

## 🎯 What is AtomIngest?

AtomIngest is a YAML-driven data ingestion framework that eliminates boilerplate code for data pipelines. Define your data schema, validation rules, and transformations in YAML - the framework handles:

- ✅ Dynamic ORM model generation
- ✅ Data validation and type conversion
- ✅ Normalization and cleaning
- ✅ Database operations (insert/update/upsert)
- ✅ Error handling and dead letter queues
- ✅ Security (PII encryption, audit trails)
- ✅ Performance optimization (batching, async)

## 💡 Why AtomIngest?

**For Fintech Teams:**
- Regulatory compliance built-in
- Immutable audit trails
- PII data protection
- High reliability and data quality

**For Developers:**
- No boilerplate code
- Type-safe data models
- Easy testing and debugging
- Extensible with hooks

## 📖 Version

This documentation is for **AtomIngest v0.1.0** (Alpha)

## 🤝 Contributing

Found an issue in the docs? Please open an issue or submit a pull request.
