# Changelog

All notable changes to CHORM will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.2.3] - 2025-12-30

### Security
- **Native SQL Parameterization**: Refactored SQL generation to use native driver parameterization. 
    - Introduced `Compiler` class to separate query structure from data.
    - `Expression.to_sql()` now delegates value handling to the driver via bind parameters.
    - `Literal` values (numbers, strings, dates) are passed as parameters, preventing SQL injection and ensuring correct type formatting.
    - Updated `Engine` and `Session` to compile statements into SQL templates and parameter dictionaries.

## [0.2.2] - 2025-12-24

### Added
- **Multi-Database Support**:
    - `__database__` attribute for `Table` classes to specify target database (e.g., `__database__ = "radar"`).
    - `TableMetadata.qualified_name` property returns `database.table` format.
    - All SQL statements (`SELECT`, `INSERT`, `UPDATE`, `DELETE`, `CREATE TABLE`) now support qualified table names.
    - `get_qualified_table_name()` helper function for resolving table names.
- **Database DDL Operations**:
    - `CREATE DATABASE` with `if_not_exists`, `engine` (Atomic, Replicated, Lazy), and `comment` options.
    - `DROP DATABASE` with `if_exists` option.
    - `create_database()` and `drop_database()` helper functions.
    - `Migration.create_database()` and `Migration.drop_database()` methods.
- **Smart DROP TABLE Protection**:
    - `Migration.drop_table()` method with automatic size protection bypass.
    - Attempts normal DROP first, retries with `max_table_size_to_drop=0` on size rejection.
    - Warning about UNDROP window (~8 minutes) when dropping large tables.
- **Migration Safety**:
    - `DROP DATABASE` is NEVER auto-generated by migration generator.
    - Warning when `DROP DATABASE default` is attempted.
    - Warning when migration generation includes DROP TABLE operations.
- **Introspection**: Generates `__database__` attribute when introspecting non-default databases.
- **Integration Tests**: New `test_database_support.py` with comprehensive database operation tests.

### Changed
- Auto-migration now uses `self.drop_table()` instead of raw SQL for better protection.
- `TableIntrospector.get_tables()` returns qualified names for non-default databases.
- `TableIntrospector.get_table_info()` accepts qualified names (e.g., `"radar.products"`).

## [0.2.1] - 2025-12-14

### Added
- **Refactored Declarative API for MVs**:
    - Support for class references in `MaterializedView(to_table=User)`.
    - `__from_table__` attribute for auto-generating simple `SELECT *` views without manual SQL.
- **Enhanced Introspection**: 
    - Generates pythonic model code using class references for Materialized Views.
    - Improved topological sorting of generated models (tables before views).
- **Pandas Integration**: Added `query_df()` to `Engine`, `Session`, `AsyncEngine`, and `AsyncSession` for direct DataFrame export (requires `pandas`).
- **Async Improvements**: Implemented lazy iteration and tuple access for `AsyncSession` results.

### Fixed
- **Testing**: Fixed mocking in `test_pandas_and_async.py` preventing accidental connection attempts during tests.

## [0.2.0] - 2025-12-13

### Added
- **Materialized Views Support**:
    - **Engine**: Added `MaterializedView` engine class.
    - **Declarative**: Added `__select__` attribute to define view queries.
    - **DDL**: Support for `CREATE MATERIALIZED VIEW`.
    - **Migrations**: Auto-migration support for creating and updating Materialized Views.
    - **Introspection**: Basic introspection of Materialized Views.
- **Performance**: Implemented lazy result iteration (`Result` object is now an iterator).
- **Testing**: Added `tests/test_materialized_views.py` and integration tests.

## [0.1.6] - 2025-12-09

### Fixed
- **CLI**: Fixed `chorm introspect` command failing with `TypeError` due to unescaped `%` characters in system table queries.
- **Tests**: Refactored Distributed table integration tests to run on a single ClickHouse node (using local loopback), removing the requirement for a second instance on port 8125.

## [0.1.5] - 2025-12-09

### Fixed
- **Regression**: Fixed `NameError: name '_escape_string' is not defined` in deprecated `BatchInsert` and `BatchUpdate` classes.

### Security
- **Critical**: Fixed SQL Injection vulnerabilities in `Insert`, `Update`, `Delete` statements where string literals were not properly escaped.
- **Critical**: Fixed SQL Injection vulnerability in legacy `BatchInsert` and `BatchUpdate` classes.
- **Medium**: Fixed Path Traversal vulnerability in migration CLI commands (`make-migration`, `auto-migrate`).
- **Medium**: Fixed potential SQL Injection in table introspection (system table queries).
- **Hardening**: Implemented password masking in `EngineConfig` string representation to prevent accidental credential leakage in logs.
- **Hardening**: Added support for parameterized queries in `Session.execute()` to encourage safe coding practices.

## [0.1.4] - 2025-12-09

### Added
- **ClickHouse Codec Support**: Added `codec` parameter to `Column` (e.g., `Column(..., codec=ZSTD(1))`).
- **SQLAlchemy-style Metadata**: Added `MetaData` registry for schema management (`metadata.create_all()`, `metadata.drop_all()`).
- **Alembic-style CLI**: 
    - Redesigned project structure: `migrations/versions/` and `migrations/env.py`.
    - `env.py` for configuring auto-migration metadata.
    - Multiple migration naming modes: `uuid` (default), `int`, `django`.
    - Configuration via `chorm.toml`.
- **Batch Optimization**: Promoted native batch insert implementation (`chorm.batch.bulk_insert`, `ClickHouseBatchInsert`) for high performance.
- **Connection Pool Liveness**: Added active connection validation and pre-pings.

### Changed
- **Auto-Migration**: Refactored `auto-migrate` to use `MetaData` registry for reliable model discovery instead of file scanning magic.
- **Performance**: Making Pandas an optional dependency (`pip install "clickhouse-chorm[pandas]"`).
- **Optimization**: Deprecated legacy SQL-based batch classes.
- **Testing**: Refactored Distributed table tests.

### Fixed
- **Testing**: Fixed CI workflows to properly wait for ClickHouse.

## [0.1.3] - 2025-12-06

### Changed
- `AggregateFunctionType` constructor now accepts `*arg_types` instead of tuple (backward compatible)
- Optimized import generation in table introspection (only imports used engines)
- Improved introspection handling for empty types and whitespace normalization

### Fixed
- Fixed `AggregateFunction` generation in introspection (removed unnecessary parentheses in argument types)
- Fixed handling of empty type strings in table introspection
- Fixed `partition_key` generation with proper quote escaping in introspection
- Fixed Distributed table introspection to handle cases when `engine_full` is missing
- Fixed handling of AggregateFunction columns with no argument types

### Added
- Support for `ReplacingMergeTree` with version column in table introspection
- Support for `groupUniqArray` and `countDistinct` aggregate functions in introspection
- Comprehensive integration tests for table introspection (`test_introspection_integration.py`)

### Testing
- Added integration tests for table introspection with MergeTree, AggregatingMergeTree, and Distributed engines

## [0.1.2] - 2025-12-05

### Added

#### Distributed Tables Support
- Full support for `Distributed` table engine
- Configuration via `Distributed(cluster, database, table, sharding_key, policy_name)`
- Support for sharding key expressions (e.g., `rand()`, column-based sharding)
- Support for storage policy parameter
- DDL generation with proper syntax for Distributed tables
- Table introspection support for Distributed tables
- Integration tests for Distributed table operations (INSERT, SELECT, CREATE)

#### AggregateFunction Support
- Full support for `AggregateFunction` data type
- Support for all aggregate function combinators:
  - State combinators: `sumState`, `avgState`, `countState`, `uniqState`, `uniqExactState`, `minState`, `maxState`
  - Merge combinators: `sumMerge`, `avgMerge`, `countMerge`, `uniqMerge`, `uniqExactMerge`, `minMerge`, `maxMerge`
  - Conditional combinators: `sumIfState`, `avgIfState`, `countIfState`, `minIfState`, `maxIfState`, `uniqIfState`, and their `Merge` counterparts
- Parameterized function support for `quantile` and `quantiles`:
  - `quantileState(level)(value)` syntax via `ParameterizedFunctionCall`
  - `quantileMerge(level)(state)` syntax
  - `quantilesState(levels)(value)` and `quantilesMerge(levels)(state)` for multiple quantiles
- Table introspection support for `AggregateFunction` columns
- Integration tests for AggregateFunction with `AggregatingMergeTree` engine
- Support for merging multiple aggregate states

### Changed
- Enhanced DDL generation to exclude unsupported clauses (PRIMARY KEY, ORDER BY, PARTITION BY, SAMPLE BY, TTL) for Distributed tables
- Improved table introspection to handle Distributed engine arguments correctly
- Window functions now support `over()` method on both `FunctionCall` and `ParameterizedFunctionCall`

### Testing
- Added integration tests for Distributed tables with cluster configuration
- Added comprehensive integration tests for AggregateFunction operations
- Updated test infrastructure to support multi-node ClickHouse cluster for Distributed table testing

## [0.1.1] - 2025-12-05

### Fixed
- Fixed test failures related to window functions (`AttributeError: 'FunctionCall' object has no attribute 'over'`)
- Fixed column mismatch issues in `INSERT FROM SELECT` statements for AggregateFunction tables
- Fixed test failures related to aggregate function merge operations
- Improved error handling in integration tests

## [0.1.0] - 2025-12-05

### Added

#### Core ORM Features
- Declarative table definitions with `Table` base class
- Column descriptors with type safety
- Support for all ClickHouse data types (21+ types)
- MergeTree family engines with full configuration
- Synchronous and asynchronous session management
- Query builder with fluent API

#### Query Construction
- SELECT queries with WHERE, ORDER BY, LIMIT, OFFSET
- ClickHouse-specific clauses: PREWHERE, FINAL, SAMPLE, SETTINGS
- JOIN support (INNER, LEFT, RIGHT, FULL, CROSS, ASOF, ARRAY JOIN)
- GROUP BY and HAVING clauses
- Subqueries and Common Table Expressions (CTEs)
- Window functions with PARTITION BY and ORDER BY
- UNION, INTERSECT, EXCEPT set operations
- DISTINCT, LIMIT BY, WITH TOTALS

#### DML Operations
- INSERT with batch support
- UPDATE (mapped to ALTER TABLE UPDATE)
- DELETE (mapped to ALTER TABLE DELETE)
- Bulk insert operations (100,000+ rows/batch)
- DataFrame integration for pandas

#### DDL Operations
- CREATE TABLE with all MergeTree engines
- DROP TABLE, TRUNCATE TABLE, RENAME TABLE
- ALTER TABLE: ADD/DROP/MODIFY/RENAME COLUMN
- Index management (minmax, set, bloom_filter, tokenbf, ngrambf)
- TTL support for data retention policies
- Partition management (DETACH, ATTACH, DROP, FETCH)
- Materialized views
- Dictionary support

#### Advanced Features
- Connection pooling (sync and async)
- Retry logic with exponential backoff
- Health checks and monitoring
- Query caching with LRU eviction
- Performance metrics collection
- Query complexity analysis
- Batch operations optimization
- Model validation system (7 validators)

#### CLI Tools
- `chorm init` - Initialize new project
- `chorm make-migration` - Generate migration files
- `chorm migrate` - Apply migrations
- `chorm show-migrations` - List migration status
- `chorm downgrade` - Rollback migrations
- `chorm introspect` - Generate models from existing tables

#### Production Features
- Comprehensive error handling with ClickHouse-specific errors
- Connection timeout configuration
- SSL/TLS support
- Proxy support
- Compression (lz4, zstd, brotli, gzip)
- Query settings presets (fast, memory_efficient, heavy_analytics, etc.)
- Execution statistics tracking

### Documentation
- Comprehensive README with examples
- Advanced analytics guide
- Connection configuration guide
- Pooling guide
- ClickHouse indexes guide
- Validation guide
- 14 example scripts

### Testing
- 628 unit and integration tests
- 60 integration tests with live ClickHouse
- Full test coverage for all major features

[0.1.3]: https://github.com/zwickvitaly/chorm/releases/tag/v0.1.3
[0.1.2]: https://github.com/zwickvitaly/chorm/releases/tag/v0.1.2
[0.1.1]: https://github.com/zwickvitaly/chorm/releases/tag/v0.1.1
[0.1.0]: https://github.com/zwickvitaly/chorm/releases/tag/v0.1.0
